Research

It’s a perfect fall Sunday morning here in Phoenix. After a brutally hot summer the air is cool, the sky is clear, and the fresh air is drifting into the hotel ballroom while I wait for my daughter to take the stage in the Irish dance regionals competition. The schedule is a little behind, so I’m sitting here on my iPad catching up on the security newsletters and posts that usually pile up during the week when I’m more focused on my own deliverables. During the week I tend to do a decent job of keeping up with the latest cloud security feature releases, but I tend to fall behind on the breach and vulnerability reports. There are… a lot of breach and vulnerability reports. One of the wild things about having started early in cloud security is that I’ve witnessed the progression from a nascent technology that neither attackers nor defenders really had a good handle on, to our current high-stakes eternal cat and mouse game. As a defender I consider it absolutely essential to read every teardown of every breach I can get my hands on (usually published by incident response service companies), while also keeping up on the latest vulnerability research. Which brings me back to the ballroom. As I close the last browser tab, the accordion and keyboardist playing an Irish reel (or maybe a jig, I really am bad at music), I realized I was instinctively mentally sorting these reports into three buckets: Yada yada “exposed credentials” yada yada a complex series of steps yada yada. Yada yada “excessive privileges” yada yada a complex series of steps yada yada. Yada yada “public facing with a known vulnerability” yada yada a complex series of steps yada yada. Of the dozen reports I’ve sorted through today, a mix of breach walkthroughs and novel attack patterns from vulnerability researchers, every single one fit into these three buckets. Like they do every week. Look, learning about more advanced attack patterns is important. Knowing how to trace and contain an incident once it progresses past the initial access is an increasingly rare and valuable skill set. And finding and fixing all the drops of misconfigurations and exposures in my buckets, at scale, ain’t easy. But it’s all too easy to get lost in the complexity, especially when you haven’t been doing this cloud security stuff for a while. Just remember, this isn’t rocket science. Stamp out static credentials, turn on MFA, and stop putting vulnerable crap on the Internet. That should keep you busy for a while. Maybe someday Chris and I will need to re-prioritize the Universal Cloud Threat Model. But today I’ll watch a little dance, enjoy a little sun, and maybe catch up on some comic books during the breaks. Share:

We have a major problem. It isn’t really getting better, and soon a critical window of opportunity will close that we can’t afford to lose. I don’t say this lightly, and I think anyone who has read my prior work knows I am not prone to FUD. No one can possibly know the actual percentage of enterprise workloads and applications that have moved to cloud, but every statistic I could find estimates that, at most, it is somewhere in the range of 25% (here’s one Gartner take). I think under 25% is likely accurate, but I estimate that well over 90% of organizations have some production workloads in cloud, including SaaS and PaaS/IaaS. The lake is wide but only deep for a relatively small number of enterprises. This is natural and expected; it takes decades to transition existing workloads, especially when they are running happily in datacenters and there’s no major driver to move them out. This is our window. Most organizations are in the shallow end of the pool, staring wistfully at the adventurous kids jumping off the high dive and frolicking around in the deep end. We have a choice — wait, learn to swim, or strap on some floaties and hope for the best. Oh, and there’s no lifeguard and there are most definitely some sharks. With lasers. If organizations don’t improve their cloud governance, they have no chance of meaningfully improving their cloud security. That’s bad enough with today’s relatively limited cloud adoption, but as we gradually move more and more workloads to the cloud, without effective governance the problem will increase exponentially. Nearly every single cloud security issue and breach is the direct result of a governance failure, not a technology failure. Cloud Governance Anti-Patterns I started working hands-on in cloud security in 2010. In any given year I probably talk with hundreds of organizations, if you include training classes and webinars. As an IANS faculty member I take, on average, 3-5 advisory calls a week, mostly with large enterprises. Each year I run multiple cloud security assessments, advisory and consulting engagements (most with larger organizations, including some of the largest in the world). I also provide advisory calls through the Cloud Security Alliance. This post is based on consistent trends I see throughout these calls, projects, and other relationships. In many calls the customer starts to describe a narrow problem which I quickly recognize is a larger governance issue. I often stop them, describe the anti-pattern, and it’s almost like I just magically described their entire childhood. It’s like my work as a paramedic: using key symptoms to identify the larger problem. This is a big dataset, and some of these issues directly contradict each other as different organizations make different mistakes on opposite sides of the spectrum: “We can’t slow down developers.” Security may be allowed to put in some basic requirements, but is often not allowed to install any significant preventative controls. They are often forced to rely on a CSPM/CNAPP and, at best, get to escalate only critical and high issues. IAM is a disaster with teams making major use of static credentials, like AWS IAM Users (which cause 66% of all AWS customer security incidents, according to AWS). “We don’t trust cloud and have to comply with our existing security policies and processes.” Security does get to slow things down, but typically lacks a sufficient technical understanding of cloud and tries to shoehorn it into existing processes. The organization tries to rely on existing security tools, and focuses too much on the network and too little on IAM. Cloud usage is so constrained that teams may just give up and keep deploying into the datacenter. “Cloud is just another datacenter.” There is little acceptance that cloud computing is a fundamentally different technology, which requires a different skillset. Neither infrastructure, development, nor security teams are effectively trained and tooled; instead they are expected to learn as they go. Many projects are just rehosted into the cloud, which reduces reliability and security while increasing costs. There are two subtypes of this pattern: “We must migrate n% of workloads by x date.” Usually driven by datacenter contract renewal dates. “We have $n credits from our vendor (usually Microsoft or Oracle), so we need to use those.” “We are going multicloud.” These organizations usually haven’t finished establishing an effective security program for one cloud, but they are going into other clouds. This is often tied to “we can’t slow down developers.” Multicloud isn’t inherently wrong, but it’s horrifically wrong without proper governance and investment in tools and people. The security teams in these organizations almost entirely rely on CSPM tools for blocking and tackling, and there is almost never investment in having at least one security subject matter expert for each cloud. There are four subtypes I often see: “We are going to be cloud agnostic and run everything in containers.” The expectations is that everything will work in containers wherever you want to deploy it, because the enterprise either thinks they can save money and dynamically move workloads wherever they are cheaper, or because developers want to use their favorite toys. For the record, I am as likely to see a living unicorn as a truly cloud-agnostic workload. “We need to backup our workloads in case our cloud provider has an outage.” If you want to completely rebuild your entire application stack on multiple platforms which don’t share any fundamental technology characteristics, be prepared to pay up. “We got some credits from $provider we need to use.” So either you lose credits, or you pay to up-skill your teams, or you… do neither, and have a poorly supported workload running on a platform on which nobody is expert. “We need to go multicloud in case $provider has an outage.” Have you tracked outages? Do you architect within your existing provider to handle outages? Executive leadership is disengaged and doesn’t set the ground rules. This one isn’t in quotes because that’s never how the

I just published a piece on Apple Intelligence at TidBITS that I’m pretty excited to release. I wrote it (literally sitting poolside on vacation) to try and explain why this matters to someone even if they don’t know anything about AI or security. For those of us in cloud security, some really interesting things are going on: This is confidential computing, but designed for a very specific purpose, which gives Apple more latitude in how they design controls. Think AWS Nitro (because this is deeper than SGX) with some metrics/monitoring to detect tampering. Apple can model and measure their workloads, and even architected a system to publicly share results, so individual devices can validate that the code is running at expected. Apple designed the system with the assumption that an advanced adversary will gain physical access to servers. That’s one hell of a threat model, and… exactly the kind of adversary Apple faces (hello, governments). The non-targeting defenses are excellent. I really appreciate two aspects: Apple can’t track requests back to individual users. They added a third-party intermediary so they never see the traffic source. If an attacker compromises a server/node, they can’t steer a user to it. This is trust but verify on steroids. Apple built a system for continuous external validation. I don’t think I’ve seen anything like it before — certainly not at scale. Lotta crypto. Like, down to the chips and digital certificates in the Secure Enclave on your phones. On the AI side, there’s some cool stuff around how they are optimizing for devices, different models, and using transformers. I also suspect they may be using RAG to interface with the on-device semantic index, but I could be wrong there. Anyway, it was a ton of fun to write. Sorry it’s so long. Read it here! Share:

The next phase of cloud security won’t be about shiny new products or services, although we’ll have those. It won’t be about stopping the next world-ending cloud 0-day, but we’ll continue trying to prevent them. It won’t be about AI, but we’ll still have to do something with AI to appease our machine overlords. It will be about making cloud deployments more inherently secure through better, smarter defaults, and better, smarter, and yes, cheaper, built-in capabilities. Here’s why: When I first started researching and working with public cloud about 15 years ago, I realized that cloud providers have massive economic incentives to be better at security than your organization. A major breach of a cloud provider that affects all (or most) tenants would be an existential event which would destroy trust in that provider and crater their business. We’ve arguably had moderate multi-tenant events, and are witnessing events in real time — wondering whether my theory will stand, and a major CSP will suffer from a direct breach (as a result of Microsoft’s recent incidents and the CISA CSRB report). This was the origin of the shared responsibilities model. There’s a waterline in the technology: below it the cloud provider is responsible for ensuring the services you consume are inherently secure. Above it you are responsible for how you secure and configure what you use. Security is transitive. When I build on a service, I am only as secure as the underlying service. It turns out this plays both ways. It’s a two-way door. Security impacts are also transitive. If a customer on a cloud platform suffers a major security breach, every headline includes the name of the cloud provider. Sure, you can blame the customer for misconfiguring your service, but that doesn’t mean everyone won’t still think you’re responsible. Thus I present the Cloud Shared Irresponsibilities Model. Cloud providers will be considered partially responsible for any customer breach involving their services, even if the breach was due to customer misconfiguration. This really hit home this week with the Ticketmaster and Santander debacle. An intel firm called Hudson Rock claimed that Snowflake was the source of the breach, and that other companies were affected. Snowflake followed up (backed by Mandiant and Crowdstrike) that the attacks targeted the breached companies and took advantage of clients with single factor authentication. While investigations are ongoing, this is negative for both the breached companies and Snowflake. (And I really wouldn’t want to be Hudson Rock right now, unless I had damn good evidence). Snowflake didn’t do anything wrong. But it kinda doesn’t matter at this point — heck, even if in a fictitious world it turns out the Ticketmaster data wasn’t even in Snowflake, no one will read the follow-up headlines. Or let’s go way back to the Capital One breach. To this day some people still think it was an insider attack by an AWS employee, or a former employee using special knowledge. Nope, it was a former employee using well-known techniques, which I even had been talking about in a training class for the prior year (we had a lab for the credential abuse part!). Here’s the messy part. AWS was partially responsible for the breach. They didn’t do certain things that could have significantly reduced the risk that Capital One would make those mistakes, or eliminated them completely. How do I know? In the years, since we’ve gotten IMDSv2 (and can now enforce it as a default), Block Public Access for S3 (and better tools to determine whether a bucket is potentially public), new regions are opt-in only, and various other enhancements. Microsoft tried to play the customer blame game, but they were hammered in that CSRB report for charging more for the security tools needed to reduce the risk of the attacks. The Shared Responsibilities Model forced providers to create secure base services, but pushed blame for misconfigurations onto customers. The Shared Irresponsibilities Model pushes negative impact back on the cloud provider for these mistakes. It’s about restoring balance to the Force. If I’m right, what will we see? Cloud providers will improve their defaults. For example, there are providers today which do not allow you to have certain accounts without MFA enabled by default (e.g., AWS is adding this requirement for root user accounts). Some security capabilities that customers pay for today will either become ‘free’ or much cheaper (e.g., Microsoft is reducing/eliminating costs for some logs, and/or extending the free retention period). More successful cloud providers will make security simpler and easier. Okay, that last one might be a stretch. It’s there to amuse my fellow cloud security professionals. I’m sure Chris is snorting some sort of not-beer out his nose right now. I really don’t think the cloud providers (other than Microsoft — seriously, read that CSRB report) have done anything wrong. It’s very hard to anticipate failure states, and to insert security which will add friction and slow down the primary buyers and users of cloud services. But now that government regulators have shifted their collective gaze, that media companies prefer headlines with ‘Amazon’, ‘Google’, and ‘Microsoft’ in them, and cloud platforms are becoming the default for new projects, it’s hard not to see this shift towards more secure cloud substrates accelerating. And I’ll finish with a simple KPI we can use to measure maturity across all platforms: The time or data volume before a cloud provider requires MFA on all administrative accounts. Share: