Version 56 (modified by robguttman, 4 years ago) (diff)


Orchestrates AWS cloud resources using boto and pychef

This plugin has been in production use since Feb 2011. I welcome others to try it out and submit tickets for bugs and new features. - robguttman, 8 June 2011



  • Launch an ec2 instance, bootstrap it with chef and apply one or more chef roles
  • Execute commands (e.g., deployments) to ec2 instances based on environment or role

There are additional features and quite possibly more to come, but the focus is on orchestrating between AWS and Chef inclusive of managing ec2 instances based on environment or role.





  1. Install the plugin (after downloading and unzipping):
    cd cloudplugin/0.12
    sudo python install


  1. Enable the plugin in trac.ini:
    cloud.* = enabled

You can alternatively use the Trac Web Admin GUI.

  1. Configure the main trac.ini section. A minimal configuration would look like this (although several are optional):
    aws_key = <aws-access-key>
    aws_secret = <aws-secret-key>
    aws_keypair = <ec2-keypair-name>
    aws_keypair_pem = /path/to/keypair.pem
    aws_username = <ec2-username>
    rds_username = <rds-master-username>
    rds_password = <rds-master-password>
    chef_base_path = /path/to/.chef
    chef_boot_run_list = <list-of-roles-or-recipes>
    jabber_server = <jabber-server>
    jabber_port = <jabber-port>
    jabber_username = <jabber-username>
    jabber_password = <jabber-password>
    jabber_channel = <jabber-channel>

AWS access configuration

The aws_* and rds_* options (see example above) are for your AWS credentials and security information. For more info, see here.

Chef access configuration


Jabber configuration

You can optionally configure notifications to jabber/xmpp for commands and environments via the jabber_* options. To enable for only deployments, for example, set a notify_jabber param in the [cloud.environments] section as follows:

notify_jabber = deploy

If you wanted to include audits, simply set the value to deploy, audit instead. To enable jabber notifications for other commands, list which ones to enable in the [cloud.commands] section:

notify_jabber = apacherestart, apachestop, apachestart

Bugs/Feature Requests

Existing bugs and feature requests for CloudPlugin are here.

If you have any issues, create a new ticket.


Download the zipped source from here.





EC2 Instances



The different configuration options are broken out below.

Labels and such

title = EC2 Instances
order = 1
label = EC2 Instance
description = AWS EC2 instances.


Field Definitions


 .. = text = Name = NameHandler
field.run_list = multiselect
field.run_list.label = Roles
field.run_list.index = role
field.run_list.handler = RunListHandler
field.ec2.placement_availability_zone = select
field.ec2.placement_availability_zone.label = Availability Zone
field.ec2.placement_availability_zone.value = us-east-1c
 .. many more fields ..

Each field.* option must map exactly to its chef attribute name. For example, ec2.placement_availability_zone above maps to the ec2 attribute namespace and the placement_availability_zone attribute within that namespace in chef. This field is defined as a select field with a display label and several options separated by a pipe (|) just like Trac custom fields. There are some important differences to Trac custom fields, however:

  • A field can define a search index instead of options
  • A field can define a "field handler"
Search indexes for options

You may prefer to define a select (or multiselect) field's options by using a chef search index (e.g., the built-in node or role, or a data bag) instead of listing them in the trac.ini file. For example, a field defined like this:

field.ec2.ami_id = select
field.ec2.ami_id.label = Image ID
field.ec2.ami_id.index = ec2_ami_id

will query chef for and use a ec2_ami_id data bag items. The individual data bag items should be formatted like this:

  "id": "ami-xxxxxxxx",
  "value": "ami-xxxxxxxx",
  "order": 1


Field handlers





crud_view = name, ec2.instance_type, ohai_time, ..
crud_new = run_list, created_by, created_at, ..
crud_edit = run_list, created_by, created_at*, ..

grid_columns = name, ec2.instance_type, ohai_time, ..
grid_sort = ohai_time
grid_asc = 0



RDS Instances

RDS instances can be created and deleted using this plugin. The rds instance resource type is configured almost identically to ec2 instances (see above). One important difference is that the list of rds instances and their attributes are saved in a chef data bag. This makes their attributes available to chef recipes. In order to ensure that this data is up-to-date, you can click on the "Audit" button in the grid view.

EIP Addresses

EIP Addresses can be created, associated with instances, and deleted using this plugin. The eip address resource type is configured identically to rds instances (see above).

EBS Volumes

EBS Volumes can be created, attached or detached to instances, and deleted using this plugin. The ebs volume resource type is configured identically to rds instances (see above). The plugin will also allow you to optionally attach an EBS volume upon launching an ec2 instance.


To help manage many ec2 instances, you can define commands to be run on them in the "Commands" section. The command resource type is configured similarly to ec2 and rds instances. Their data is maintained in a chef data bag. An example command may look this this:

/usr/bin/ssh -i %(keypair_pem)s root@%(host)s "service apache2 restart"

The %(keypair_pem)s will get replaced with the aws_keypair_pem as defined in trac.ini's [cloud] section (see above). Also, %(host)s will get replaced with the public DNS name of each host. Hosts are selected by environment and/or role.


With Chef 0.10 comes support for environments. This plugin builds on this support by allowing (read: presuming) that AWS resources belong to one and only one environment. For example, there's a default environment field that can get set when creating a new ec2 or rds instance. The environment resource type also allows you to define and set attributes per environment via the plugin's webui. For example, you can set different version control repo branch names and/or changesets per environment so that the right source code gets deployed to each environment.

Deploy and Audit

There are two special commands that can be defined: "deploy" and "audit". A command with a "deploy" ID will cause a "Deploy" button to appear in an environment's View along with widgets to select which role(s) should be used to identify the instances to which to deploy. The 'deploy' script itself is left for you to define as such scripts often require significant logic specific to your system.

Similarly, a command with an "audit" ID will cause an "Audit" button to appear in an environment's View along with widgets to select which role(s) should be used to identify the instances of which to audit. The 'audit' script itself is left for you to define as such scripts often require significant logic specific to your system. If a JSON list of HTML strings is returned, then these HTML snippets are concatenated at the end of each listed node in the progress view (see below) thus providing the audit details. For example, you can return links to the diffs between the target repo changeset and the node's current repo changeset (which is how we use this audit feature).

Progress views

Actions such as launching or bootstrapping an ec2 instance or deploying code to an environment can take a variably long time - usually longer than the default Apache timeouts (e.g., FcgidIOTimeout). Instead of wrestling with Apache timeout configurations, this plugin executes each of these actions in a separate, spawned process (an orphaned daemon) that survives Apache timeouts and restarts. Surviving Apache restarts is also helpful when Trac itself is chef-managed and may be restarted by the chef-client upon config changes. Creating a new ec2 instance, executing remote commands, etc. now brings up a new 'Progress' page that tracks the spawned process' progress:

For launching ec2 instances, as the instance id and public DNS become known, they're presented. The progress view is also used for other (potentially) long-lived actions such as deployments and audits where the list of nodes is shown with the same progress icons and time durations.

Recent Changes

[14004] by rjollos on 2014-07-12 20:27:48
Correct license keyword argument in Refs #11832.
[13996] by rjollos on 2014-06-25 02:56:32
Fixed author in COPYING file. Refs #11832.
[13994] by rjollos on 2014-06-25 02:51:36


Author: robguttman
Maintainer: robguttman

Attachments (4)

Download all attachments as: .zip