I’ve been teaching cloud incident response with Will Bengtson at Black Hat for a few years now, and one of the cool side effects of running training classes is that we are forced to document our best practices and make them simple enough to explain. (BTW — you should definitely sign up for the 2024 version of our class before the price goes up!) One of the more amusing moments was the first year we taught the class, when I realized I was trying to hand-write all the required CloudTrail log queries in front of the students, because I had only prepared a subset of what we needed. As I wrote in my RECIPE PICKS post, you really only need a handful of queries to find 90% of what you need for most cloud security incidents. Today I want to tie together the RECIPE PICKS mnemonic with the sample queries we use in training. I will break this into two posts — today I’ll load up the queries, and in the next post I’ll show a sample analysis using them. A few caveats: These queries are for AWS Athena running on top of CloudTrail. This is the least common denominator — anyone with an AWS account can run Athena. You will need to adapt them if you use another tool or a SIEM, but those should just be syntactical changes. Obviously you’ll need to do more work for other cloud providers, but this information is available on every major platform. These are only the queries we run on the CloudTrail logs. RECIPE PICKS includes other information these queries don’t cover, or that don’t cleanly match a single query. I’ll write other posts over time, showing more examples of how to gather that data, but none of it takes very long. In class we spend a lot of time adjusting the queries for different needs. For example, when searching for entries on a resource you might need to look in responseElements or requestParameters. I’ll try to knock out a post on how that all works soon, but the TL;DR is: sometimes the ID you query on is used in the API call (request); other times you don’t have it yet, and AWS returns it in the response. RECIPE PICKS is not meant to be done in order. It’s similar to a lot of mnemonics I use in paramedic work. It’s to make sure you don’t miss anything, not an order of operations. With that out of the way, here’s a review of RECIPE PICKS (Canva FTW): Now let’s go through the queries. Remember, I’ll have follow-on posts with more detail — this is just the main reference post to get started. A few things to help you understand the queries: For each of these queries, you need to replace anything between <> with your proper identifiers (and strip out <>). A “%” is a wildcard in SQL, so just think “*” in your head and you’ll be just fine. You’ll see me pulling different information on different examples (e.g., event name). In real life you might want to pull all the table fields, or different fields. In class we play around with different collection and sorting options for the specific incident, but that is too much for a single blog post. Resource If I have a triggering event associated with a resource, I like to know its current configuration. This is largely to figure out whether I need to stop the bleed and take immediate action (e.g., if something was made public or shared to an unknown account). There is no single query because this data isn’t in CloudTrail. You can review the resource in the console, or run a describe/get/list API call. Events Gather every API call involving a resource. This example is for a snapshot, based on the snapshot ID: SELECT useridentity.arn, eventname, sourceipaddress, eventtime, resources FROM <your table name> WHERE requestparameters OR responseelements like ‘%<snapshot id>%’ ORDER BY eventtime Changes Changes is a combination of the before and after state of the resource, and the API call which triggered the change associated with the incident. This is another one you can’t simply query from CloudTrail, and you won’t have a change history without the right security controls in place. This is either: AWS Config A CSPM/CNAPP with historical inventory A cloud inventory tool (if it has a history) Many CSPM/CNAPP tools include a history of changes. This is the entire reason for the existence of AWS Config (well, based on the pricing there may be additional motivations). My tool (FireMon Cloud Defense) auto-correlates API calls with a change history, but if you don’t have that support in your tool you may need to do a little manual correlation. If you don’t have a change history this becomes much harder. Worst case: you read between the lines. If an API call didn’t error, you can assume the requested change went through and then figure out the state. Identity Who or what made the API call? CloudTrail stores all this in the useridentity element, which is structured as: useridentity STRUCT< type:STRING, principalid:STRING, arn:STRING, accountid:STRING, invokedby:STRING, accesskeyid:STRING, userName:STRING, sessioncontext:STRUCT< attributes:STRUCT< mfaauthenticated:STRING, creationdate:STRING>, sessionissuer:STRUCT< type:STRING, principalId:STRING, arn:STRING, accountId:STRING, userName:STRING>, ec2RoleDelivery:string, webIdFederationData:map<string,string> > The data you’ll see will vary based on the API call and how the entity authenticated. Me? I keep it simple at this point, and just query useridentity.arn as shown in the query above. This provides the Amazon Resource Name we are working with. Permissions What are the permissions of the calling identity? This defines the first part of the IAM blast radius, which is the damage it can do. The API calls are different between user and role, and here’s a quick CLI script that can pull IAM policies. But if you have console access that may be easier: #!/bin/bash # Function to get policies attached to a user get_user_policies() { local user_arn=$1 local user_name=$(aws iam get-user –user-name $(echo $user_arn | awk -F/ ‘{print $NF}’) –query ‘User.UserName’ –output text) echo “User Policies for $user_name:”
