Blog

Summary: News…. and pulling an AMI from Packer and Jenkins

By Rich

Rich here.

Before I get into tech content, a quick personal note. I just signed up for my first charity athletic event, and will be riding 250 miles in 3 days to support challenged athletes. I’ve covered the event costs, so all donations go right to the cause. Click here if you are interested in supporting the Challenged Athletes Foundation (and my first attempt at fundraising since I sold lightbulbs for the Boy Scouts. Seriously. Lightbulbs. Really crappy ones which burned out in months, making it very embarrassing to ever hit that neighborhood again. Then again, that probably prepared me for a career in security sales).

Publishing continues to be a little off here at Securosis as we all wrestle with summer vacations and work trips. That said, instead of the Tool of the Week I’m going with a Solution of the Week that s time, because I ran into what I think is a common technical issue I couldn’t find covered well anyplace else.

With that, let’s jump right in…

Top Posts for the Week

Solution of the Week

As I was building the deployment pipeline lab for our cloud security training at Black Hat, I ran into a small integration issue that I was surprised I could not find documented anyplace else. So I consider it my civic duty to document it here.

The core problem comes when you use Jenkins and Packer to build Amazon Machine Images (AMIs). I previously wrote about Jenkins and Packer. The flow is that you make a code (or other) change, which triggers Jenkins to start a new build, which uses Packer to create the image. The problem is that there is no built-in way to pull the image ID out of Packer/Jenkins and pass it on to the next step in your process.

Here is what I came up with. This won’t make much sense unless you actually use these tools, but keep it as a reference in case you ever go down this path. I assume you already have Jenkins and Packer working together.

When Packer runs it outputs the image ID to the console, but that isn’t made available as a variable you can access in any way. Jenkins is also weird about how you create variables to pass on to other build steps. This process pulls the image ID from the stored console output, stores it in a file in the workspace, then allows you to trigger other builds and pass the image ID as a parameter.

  • Install the following additional plugins in Jenkins:
    • Post-Build Script Plugin
    • Parameterized Trigger plugin
  • Get your API token for Jenkins by clicking on your name > configure.
  • Make sure your job cleans the workspace before each build (it’s an environment option).
  • Create a post-build task and choose “Execute a set of scripts”.
  • Adjust the following code and replace the username and password with your API credentials. Then paste it into the “Execute Shell” field. This was for a throwaway training instance I’ve already terminated so these embedded credentials are worthless. Give me a little credit please:

    wget –auth-no-challenge –user= –password= http://127.0.0.1:8080/job/Website/lastBuild/consoleText export IMAGE_ID=$(grep -P -o -m 1 ‘(?<=AMI:\s)ami-.{8}’ consoleText) echo IMAGE_ID=$IMAGE_ID >> params.txt

The wget calls the API for Jenkins, which provides the console text, which includes the image ID (which we grep out). Jenkins can run builds on slave nodes, but the console text is stored on the master, which is why it isn’t directly accessible some other way.

  • The image ID is now in the the params.txt file in the workspace, so any other post build steps can access it. If you want to pass it to another job you can use the Parameterized Trigger plugin to pass the file. In our training class we add other AWS-specific information in that file to run automated deployment using some code I wrote for rolling updates.

This isn’t hard, and I saw blog posts saying “pull it from the console text”, but without any specifics of how to access the text or what to do with the ID afterwards so you can access it in other post-build steps or jobs. In our case we do a bunch more, including launching an instance from the image for testing with Gauntlt, and then the rolling update itself if all tests pass.

Securosis Blog Posts This Week

Other Securosis News and Quotes

Training and Events

No Related Posts
Comments

If you like to leave comments, and aren’t a spammer, register for the site and email us at info@securosis.com and we’ll turn off moderation for your account.