Plan policy

Purpose

Plan policies are evaluated during a planning phase after vendor-specific change preview command (eg. terraform plan) executes successfully. The body of the change is exported to JSON and parts of it are combined with Spacelift metadata to form the data input to the policy.

Plan policies are the only ones that have access to the actual changes to the managed resources, so this is probably the best place to enforce organizational rules and best practices as well as do automated code review. There are two types of rules here that Spacelift will care about: deny and warn. Each of them must come with an appropriate message that will be shown in the logs. Any deny rules will print in red and will automatically fail the run, while warn rules will print in yellow and will at most mark the run for human review if the change affects the tracked branch and the Stack is set to autodeploy.

Here is a super simple policy that will show both types of rules in action:

package spacelift

deny["you shall not pass"] {
  true # true means "match everything"
}

warn["hey, you look suspicious"] {
  true
}

Let's create this policy, attach it to a Stack and take it for a spin by triggering a run:

Yay, that works (and it fails our plan, too), but it's not terribly useful - unless of course you want to block all changes to your Stack in a really clumsy way. Let's look a bit deeper into the document that each plan policy receives, two possible use cases - rule enforcement and automated code review - and some cookbook examples.

Data input

This is the schema of the data input that each policy request will receive:

{
  "spacelift:": {
    "commit": {
      "author": "string - GitHub login if available, name otherwise",
      "branch": "string - branch to which the commit was pushed",
      "created_at": "number  - creation Unix timestamp in nanoseconds",
      "message": "string - commit message"
    },
    "request": {
      "timestamp_ns": "number - current Unix timestamp in nanoseconds"
    },
    "run": {
      "based_on_local_workspace": "boolean - wether the run stems from a local preview",
      "changes": [
        {
          "action": "string enum - added | changed | deleted",
          "entity": {
            "address": "string - full address of the entity",
            "name": "string - name of the entity",
            "type": "string - full resource type or \"output\" for outputs"
          },
          "phase": "string enum - plan | apply"
        }
      ],
      "created_at": "number - creation Unix timestamp in nanoseconds",
      "runtime_config": {
        "before_init": ["string - command to run before run initialization"],
        "project_root": "string - root of the Terraform project",
        "runner_image": "string - Docker image used to execute the run",
        "terraform_version": "string - Terraform version used to for the run"
      },
      "triggered_by": "string or null - user or trigger policy who triggered the run, if applicable",
      "type": "string - PROPOSED or TRACKED",
      "updated_at": "number - last update Unix timestamp in nanoseconds",
      "user_provided_metadata": ["string - blobs of metadata provided using spacectl or the API when interacting with this run"]
    },
    "stack": {
      "administrative": "boolean - is the stack administrative",
      "autodeploy": "boolean - is the stack currently set to autodeploy",
      "branch": "string - tracked branch of the stack",
      "labels": ["string - list of arbitrary, user-defined selectors"],
      "name": "string - name of the stack",
      "repository": "string - name of the source GitHub repository",
      "state": "string - current state of the stack",
      "terraform_version": "string or null - last Terraform version used to apply changes"
    }
  },
  "terraform": {
    "resource_changes": [
      {
        "address": "string - full address of the resource, including modules",
        "type": "string - type of the resource, eg. aws_iam_user",
        "locked_by": "optional string - if the stack is locked, this is the name of the user who did it",
        "name": "string - name of the resource, without type",
        "namespace": "string - repository namespace, only relevant to GitLab repositories",
        "project_root": "optional string - project root as set on the Stack, if any",
        "provider_name": "string - provider managing the resource, eg. aws",
        "change": {
          "actions": ["string - create, update, delete or no-op"],
          "before": "optional object - content of the resource",
          "after": "optional object - content of the resource"
        }
      }
    ],
    "terraform_version": "string"
  }
}

String Sanitization

String properties in "before" and "after" objects will be sanitized in order to protect secret values. Sanitization hashes the value and takes the last 8 bytes of the hash.

If you need to compare a string property to a constant, you can use the sanitized(string) helper function.

deny["must not target the forbidden endpoint: forbidden.endpoint/webhook"] {
  resource := input.terraform.resource_changes[_]

  actions := {"create", "delete", "update"}
  actions[resource.change.actions[_]]

  resource.change.after.endpoint == sanitized("forbidden.endpoint/webhook")
}

Use cases

Since plan policies get access to the changes to your infrastructure that are about to be introduced, they are the right place to run all sorts of checks against those changes. We believe that there are two main use cases for those checks - hard rule enforcement preventing shooting yourself in the foot and automated code review that augments human decision-making.

Organizational rule enforcement

In every organization, there are things you just don't do. Hard-won knowledge embodied by lengthy comments explaining that if you touch this particular line the site will go hard down and the on-call folks will be after you. Potential security vulnerabilities that can expose all your infra to the wrong crowd. Spacelift allows turning them into policies that simply can't be broken. In this case, you will most likely want to exclusively use deny rules.

Let's start by introducing a very simple and reasonable rule - never create static AWS credentials:

package spacelift

# Note that the message here is dynamic and captures resource address to provide
# appropriate context to anyone affected by this policy. For the sake of your
# sanity and that of your colleagues, please a
deny[sprintf(message, [resource.address])] {
  message := "static AWS credentials are evil (%s)"

  resource := input.terraform.resource_changes[_]
  resource.change.actions[_] == "create"

  # This is what decides whether the rule captures a resource.
  # There may be an arbitrary number of conditions, and they all must
  # succeed for the rule to take effect.
  resource.type == "aws_iam_access_key"
}

Here's a minimal example of this rule in the Rego playground.

If that makes sense, let's try defining a policy that implements a slightly more sophisticated piece of knowledge - that when some resources are recreated, they should be created before they're destroyed or an outage will follow. We found that to be the case with the aws_batch_compute_environment, among others. So here it is:

package spacelift

# This is what Rego calls a set. You can add further elements to it as necessary.
always_create_first := { "aws_batch_compute_environment" }

deny[sprintf(message, [resource.address])] {
  message  := "always create before deleting (%s)"
  resource := input.terraform.resource_changes[_]
  
  # Make sure the type is on the list.
  always_create_first[resource.type]
  
  some i_create, i_delete
  resource.change.actions[i_create] == "create"
  resource.change.actions[i_delete] == "delete"
 
  
  i_delete < i_create
}

Here's the obligatory Rego playground example.

While in most cases you'll want your rules to only look at resources affected by the change, you're not limited to doing so. You can also look at all resources and force teams to remove certain resources. Here's an example - until all AWS resources are removed all in one go, no further changes can take place:

package spacelift

deny[sprintf(message, [resource.address])] {
  message  := "we've moved to GCP, find an equivalent there (%s)"
  resource := input.terraform.resource_changes[_]
  
  resource.provider_name == "aws"
  
  # If you're just deleting, all good.
  resource.change.actions != ["delete"]
}

Feel free to play with a minimal example of this policy in the Rego playground.

Automated code review

OK, so rule enforcement is very powerful, sometimes you want something more sophisticated. Instead of (or in addition to) enforcing hard rules, you can use plan policy rules to help humans understand changes better, and make informed decisions what looks good and what does not. This time we'll be adding more color to our policies and start using warn rules in addition to deny ones we've already seen in action.

You've already seen warn rules in the first section of this article but here it is in action again:

It won't fail your plan and it looks relatively benign, but this little yellow note can provide great help to a human reviewer, especially when multiple changes are introduced. Also, if a stack is set to autodeploy, the presence of a single warning is enough to flag the run for a human review.

The best way to use warn and deny rules together depends on your preferred Git workflow. We've found short-lived feature branches with Pull Requests to the tracked branch to work relatively well. In this scenario, the type of the run is important - it's PROPOSED for commits to feature branches, and TRACKED on commits to the tracked branch. You will probably want at least some of your rules to take that into account and use this mechanism to balance comprehensive feedback on Pull Requests and flexibility of being able to deploy things that humans deem appropriate.

As a general rule when using plan policies for code review, deny when run type is PROPOSED and warn when it is TRACKED. Denying tracked runs unconditionally may be a good idea for most egregious violations for which you will not consider an exception, but when this approach is taken to an extreme it can make your life difficult.

We thus suggest that you at most deny when the run is PROPOSED, which will send a failure status to the GitHub commit but will give the reviewer a chance to approve the change nevertheless. If you want a human to take another look before those changes go live, either set stack autodeploy to false, or explicitly warn about potential violations. Here's an example of how to reuse the same rule to deny or warn depending on the run type:

package spacelift

proposed := input.spacelift.run.type == "PROPOSED"

deny[reason] { proposed; reason := iam_user_created[_] }
warn[reason] { not proposed; reason := iam_user_created[_] }

iam_user_created[sprintf("do not create IAM users: (%s)", [resource.address])] {
  resource := input.terraform.resource_changes[_]
  resource.change.actions[_] == "create"
  resource.type == "aws_iam_user"
}

Predictably, this fails when committed to a non-tracked (feature) branch:

...but as a GitHub repo admin you can still merge it if you've set your branch protection rules accordingly:

Cool, let's merge it and see what happens:

Cool, so the run stopped in its tracks and awaits human decision. At this point we still have a choice to either confirm or discard the run. In the latter case, you will likely want to revert the commit that caused the problem - otherwise all subsequent runs will be affected.

The minimal example for the above rule is available in the Rego playground.

Cookbook

Below are a few examples of policies that accomplish some common business goals. Feel free to copy them verbatim, or use them as an inspiration.

Require human review when resources are deleted or updated

Adding resources may ultimately cost a lot of money but it's generally pretty safe from an operational perspective. Let's use a warn rule to allow changes with only added resources to get automatically applied, and require all others to get a human review:

package spacelift

warn[sprintf(message, [action, resource.address])] {
  message := "action '%s' requires human review (%s)"
  review  := {"update", "delete"}

  resource := input.terraform.resource_changes[_]
  action   := resource.change.actions[_]

  review[action]
}

Here's a minimal example to play with.

Automatically deploy changes from selected individuals

Sometimes there are folks who really know what they're doing and changes they introduce can get deployed automatically, especially if they already went through code review. Below is an example that allows commits from whitelisted individuals to be deployed automatically (assumes Stack is set to autodeploy):

package spacelift

warn[sprintf(message, [author])] {
  message     := "%s is not on the whitelist - human review required"
  author      := input.spacelift.commit.author
  whitelisted := { "alice", "bob", "charlie" }
  
  not whitelisted[author]
}

Here's the playground example for your enjoyment.

Require commits to be reasonably sized

Massive changes make reviewers miserable. Let's automatically fail all changes that affect more than 50 resources. Let's also allow them to be deployed with mandatory human review nevertheless:

package spacelift

proposed := input.spacelift.run.type == "PROPOSED"

deny[msg] { proposed; msg := too_many_changes[_] }
warn[msg] { not proposed; msg := too_many_changes[_] }

too_many_changes[msg] {
  threshold := 50

  res := input.terraform.resource_changes
  ret := count([r | r := res[_]; r.change.actions != ["no-op"]])
  msg := sprintf("more than %d changes (%d)", [threshold, ret])
  
  ret > threshold
}

Here's the above example in the Rego playground, with the threshold set to 1 for simplicity.

Back-of-the-envelope blast radius

This is a fancy contrived example building on top of the previous one. However, rather than just looking at the total number of affected resources, it attempts to create a metric called a "blast radius" - that is how much the change will affect the whole stack. It assigns special multipliers to some types of resources changed and treats different types of changes differently: deletes and updates are more "expensive" because they affect live resources, while new resources are generally safer and thus "cheaper". As per our automated code review pattern, we will fail Pull Requests with changes violating this policy, but require human action through warnings when these changes hit the tracked branch.

package spacelift

proposed := input.spacelift.run.type == "PROPOSED"

deny[msg] { proposed; msg := blast_radius_too_high[_] }
warn[msg] { not proposed; msg := blast_radius_too_high[_] }

blast_radius_too_high[sprintf("change blast radius too high (%d/100)", [blast_radius])] {
	blast_radius := sum([blast |
    					 resource := input.terraform.resource_changes[_];
                         blast := blast_radius_for_resource(resource)])
                         
	blast_radius > 100    
}

blast_radius_for_resource(resource) = ret {
	blasts_radii_by_action := { "delete": 10, "update": 5, "create": 1, "no-op": 0 }
    
    ret := sum([value | action := resource.change.actions[_]
                    action_impact := blasts_radii_by_action[action]
                    type_impact := blast_radius_for_type(resource.type)
                    value := action_impact * type_impact])
}

# Let's give some types of resources special blast multipliers.
blasts_radii_by_type := { "aws_ecs_cluster": 20, "aws_ecs_user": 10, "aws_ecs_role": 5 }

# By default, blast radius has a value of 1.
blast_radius_for_type(type) = 1 {
    not blasts_radii_by_type[type]
}

blast_radius_for_type(type) = ret {
    blasts_radii_by_type[type] = ret
}

You can play with a minimal example of this policy in The Rego Playground.

Cost management

Thanks to our Infracost integration, you can take cost information into account when deciding whether to ask for human approval or to block changes entirely.