Creating blueprints

If you have not already done so, read the Structure of a cloudlet page.

Overview

A cloudlet typically contains multiple blueprints encompassing different deployment scenarios. For example, a development blueprint might spin up a single all-in-one instance, while a staging blueprint could spin up a complete stack with multiple components, all contained within a private network.

Regardless of the blueprint chosen, the same payload data is sent to every instance. Thus it is important to structure both your blueprint template and the payload to be environmentally aware. More on how environment data is passed to the blueprint and payload is discussed later.

Responsible Blueprint Creation

Nepho 1.0 and higher are intentionally hands-off about how you build your blueprint. The reason we removed much of the blueprint construction functionality from Nepho core was to encourage faster and easier blueprint creation, especially in edge cases that the developers did not anticipate. As a cloudlet and blueprint creator/maintainer, it is your responsibility to follow Nepho conventions and best practices to ensure interoperability and smooth functioning, especially as Nepho core is updated.

Components

The blueprint.yaml file

Each blueprint is defined in a YAML file located within the cloudlet's "blueprints" directory and named with the name of the blueprint, lowercased, with spaces replaced with dashes. Here are the blueprints included in the nepho-example cloudlet:

nepho-example
└── blueprints
    ├── aws-single-host.yaml
    └── vagrant-single-host.yaml

This file contains a one line summary (shown in search results), a longer description, a provider, and zero or more parameters. Parameters are written in CamelCase. Here is an example:

--- # aws-single-host
summary: A single Amazon Linux host on a public IP
description: >
  This simple example creates a single public Amazon Linux instance on EC2 using
  CloudFormation, bootstraps Puppet, and performs a few minor configuration
  actions.
provider: aws
parameters:
  InstanceType: m1.small

Parameter values are considered the "default" and can be overridden by the user. The value can also be left blank, the parameter will still be required and will have no default.

The infrastructure template file

The blueprint template is the file that defines all the infrastructure. For the aws provider this file is a CloudFormation template (always named cf.json). For the vagrant provider this file is a Vagrantfile. The location of the template file is under the resources directory in a directory corresponding to the name of the blueprint:

nepho-example
└── resources
    ├── aws-single-host
    │   └── cf.json
    ├── common
    │   └── aws
    └── vagrant-single-host
        └── Vagrantfile

Additional files can be included within the directory alongside the infrastructure template file, Nepho will ignore them. Nepho offers two ways of running infrastructure templates:

• Unmodified - You provide an infrastructure template in its completed form. It is constructed by any appropriate process outside of Nepho, such as being written by hand, generated from a script, or assembled through a pre-processor. Nepho runs this template without any modification.
• Jinja2 - You provide an infrastructure template that contains snippets, macros, or other markup in the Jinja2 templating language. Nepho creates the final infrastructure template at stack creation time by running your file through the Jinja2 interpreter along with the stack context.

The advantage of the unmodified format is that it is more portable, and you can generate it through any process you choose, without being tied to Nepho or Jinja2. The template may be clearer and more obvious, and easier to modify.

The advantage of the Jinja2 format is that you can construct your template dynamically from building blocks and take advantage of macros to automate tedious tasks. The template may be simpler, you can incorporate comments (not allowed in JSON), and users of your cloudlet may be more able to tweak the template for their uses.

Using the Jinja2 format

All templates are processed through Jinja2 by default. Simply include Jinja2 syntax in your template. The Jinja2 environment will include in its search path the directory containing your infrastructure template as well as a second optional "common" directory corresponding to your provider, such as resources/common/aws. This second directory is a good location to target a Git submodule pointing to a library of template objects, such as our provided aws-template-library.

Here is an example of using Jinja2 markup to simplify a CloudFormation template, note the use of Jinja2 includes and macros:

{% import 'Resources/Host.json' as host with context %}
{
  AWSTemplateFormatVersion : 2010-09-09,
  Description : Simple Host Deployment,
  Parameters : {
    {% include 'Parameters/General-Instance-Parameters.json' %},
  },
  Mappings : {
    {% include 'Mappings/AWS-Instance-Mappings.json' %}
  },
  Resources : {
    {{ host.host( name='SingleServer', ports=[80, 443]) }}
  },
  Outputs : {
  }
}

This is just to give you an idea, for a working example see the nepho-example cloudlet.

Using Parameters

When a new stack is spun up Nepho passes an amalgamation of all necessary parameters by running a union of all parameters required by the provider, cloudlet, and blueprint, in that order of inheritance. In order to use a parameter in your template, it must be specified at one of those levels.

• The first layer is parameters required by the provider, these must be present for any blueprint across any cloudlet that is using the given provider. For example, the aws provider requires a parameter called AWSAccessKeyID, which it needs in order to connect to AWS.
• The second layer of parameters are those specified in the parameters block of the cloudlet.yaml file. These parameters are required for all blueprints within a given cloudlet.
• The third layer of parameters are specified at the blueprint level in the parameters block of the blueprint.yaml file. These may be the same as those specified in the cloudlet.yaml, in which case the values in blueprint.yaml will override.

This can all be a bit confusing. There is a helpful Nepho command that will display the complete context for a given blueprint at the current point in time. Run nepho stack show-context <cloudlet> <blueprint> to view the output, which will look like this:

$ nepho stack show-context nepho-example aws-single-host
Cloudlet:
  format            : 1
  name              : nepho-example
  path              : /Users/zeno/.nepho/cloudlets/nepho-example
  version           : 1.0.0
Blueprint:
  name              : aws-single-host
  provider          : aws
Parameters:
  AWSAccessKeyID    : ABC123ABC123ABC123ABC
  AWSRegion         : us-east-1
  AWSSecretAccessKey: abcdefghijklmnopqrstuvwxyz1234567890ABCD
  InstanceType      : m1.small
  KeyName           : cloudhacks-agperson

Some of the parameters listed above are defaults specified in either the provider, cloudlet.yaml, or blueprint.yaml, others are provided by the user running nepho parameter set <key> <value>.

To use parameters with CloudFormation, you must specify them in the "Parameters:" section. The Nepho AWS provider passes the parameters to CloudFormation along with the infrastructure template at stack creation, and CloudFormation interpolates the values.

To use parameters with Vagrant, you must "catch" them yourself within the Vagrantfile. The nepho-example cloudlet demonstrates how to do this by checking for environment variables named NEPHO_ParameterName, and providing defaults in case they are unset.

Note: CloudFormation will throw an error if a parameter is passed in that is not specified in the template. For this reason, you must define all possible parameters, even if you are not going to use them elsewhere. This includes parameters set at the provider level and used internally by Nepho, such as AWSRegion that you may not need. Even if you do not use these parameters, you must specify them in your infrastructure template for CloudFormation, as we do in the nepho-example blueprints. This limitation does not exist with Vagrant.

Handling Payloads

As noted above, the same payload data is sent to every instance regardless of blueprint (more information about payloads can be found in Structure of a cloudlet.) Your blueprint must properly manage the payload. The nepho-example blueprints demonstrate how to:

• Get the payload onto each instance and expand it into a standard location
• Capture parameters from the context and inject them into each instance as environment variables prefixed with NEPHO_.
• Fire standard hook scripts at set Nepho lifecycle events.

The only requirement of the payload directory is that it contain stubs of each hook script. Additional configuration management and deployment processes are entirely optional and may be handled outside of Nepho if desired.

An important aspect of crafting your payload is taking advantage of the Nepho passed parameters for determining how to configure the instance. The nepho-example cloudlet demonstrates how to translate the NEPHO_ variables into Hieradata suitable for accessing in Puppet.