Shortcuts for AWS Infrastructure Automation

AWS infrastructure automation is time consuming and at certain times the tooling documenation like cloudformation and terraform does not have clear documentation of the attributes. But for certain AWS services like the ones I'll be discussing in this post you can use the AWS Console UI to guide to create a version of the object and then grab the attributes from the object through the cli and convert it to cloudformation or terraform then you can use variable references instead of hard coding to generalize the object.

AWS Codepipeline Project

I'll start by assuming a codepipeline project was created in the console and was named example-pipeline, and you have aws cli v2 installed and creds configured.

  • First we get the pipeline information and store it.
aws2 codepipeline get-pipeline \
  --name frontend-pipeline-prod-from-production \
  --output yaml > example_pipeline.yaml

And this is a sample content of the example_pipeline.yaml file:

metadata:
  created: '2019-11-22T16:32:48.387000+01:00'
  pipelineArn: arn:aws:codepipeline:<region>:<account_id>:example-pipeline
  updated: '2019-11-22T16:42:02.548000+01:00'
pipeline:
  artifactStore:
    location: example-artifact-bucket
    type: S3
  name: example-pipeline
  roleArn: arn:aws:iam::<account_id>:role/codepipeline-role
  stages:
  - actions:
    - actionTypeId:
        category: Source
        owner: ThirdParty
        provider: GitHub
        version: '1'
      configuration:
        Branch: master
        OAuthToken: '****'
        Owner: foo
        PollForSourceChanges: 'true'
        Repo: example-repo
      inputArtifacts: []
      name: Source
      outputArtifacts:
      - name: code
      runOrder: 1
    name: Source
  - actions:
    - actionTypeId:
        category: Build
        owner: AWS
        provider: CodeBuild
        version: '1'
      configuration:
        EnvironmentVariables: "[{\n\t\"name\": \"ENVIRONMENT\",\n\t\"value\": \"prod\",\n\t\"type\":
          \"PLAINTEXT\"\n}]\n"
        ProjectName: sample-build
      inputArtifacts:
      - name: code
      name: Build
      outputArtifacts: []
      runOrder: 2
    name: deploy-step
  version: 3
  • We can convert this to cloudformation like this since all the attributes are just there as in this sample snippet:
Type: AWS::CodePipeline::Pipeline
Properties:
  ArtifactStore:
    Location: example-artifact-bucket
    Type: S3
  Name: example-pipeline
  RoleArn: arn:aws:iam::<account_id>:role/codepipeline-role
  Stages:
  - Actions:
    - ActionTypeId:
        Category: Source
        Owner: ThirdParty
        Provider: GitHub
        Version: '1'
      Configuration:
        Branch: master
        OAuthToken: '****'
        Owner: foo
        PollForSourceChanges: 'true'
        Repo: example-repo
      InputArtifacts: []
      Name: Source
      OutputArtifacts:
      - Name: code
      RunOrder: 1
    Name: Source
  - Actions:
    - ActionTypeId:
        Category: Build
        Owner: AWS
        Provider: CodeBuild
        Version: '1'
      Configuration:
        EnvironmentVariables: "[{\n\t\"name\": \"ENVIRONMENT\",\n\t\"value\": \"prod\",\n\t\"type\":
          \"PLAINTEXT\"\n}]\n"
        ProjectName: sample-build
      InputArtifacts:
      - Name: code
      Name: Build
      OutputArtifacts: []
      RunOrder: 2
    Name: deploy-step

As could be seen from the conversion all you need to do is change the attribute names from camel case to pascal to fit them as a cloudformation resource, the next step would be to change certain values to variables to generalize the template but that's case specific and could be left for the reader.

AWS ApiGateway

I'll start by assuming an api gateway is created an deployed using the console and has this id "fffffffffd" and this stage name "test", and you have aws cli v2 installed and creds configured.

  • This one is simpler, all you need to do is grab the extended swagger and drop it inside the cloudformation template and create variables/generalize.
aws2 apigateway get-export \
 --parameters extensions='apigateway' \
 --accepts application/yaml \
 --rest-api-id fffffffffd \
 --stage-name test \
 --export-type swagger \
 ./the_swagger.yaml

And this is a sample swagger output:

---
swagger: "2.0"
info:
  version: "2020-01-09T06:49:46Z"
  title: "example-api"
host: "example.com"
schemes:
- "https"
paths:
  /the-example:
    post:
      responses: {}
      x-amazon-apigateway-integration:
        uri: "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:<region>:<account_id>:function:example-func/invocations"
        passthroughBehavior: "when_no_match"
        httpMethod: "POST"
        type: "aws_proxy"
  • Now just add this to the cloudformation template, for example:
Type: AWS::ApiGatewayV2::Api
Properties:
  BasePath: "/foo"
  Body:
    swagger: "2.0"
    info:
      version: "2020-01-09T06:49:46Z"
      title: "example-api"
    host: "example.com"
    schemes:
    - "https"
    paths:
      /the-example:
        post:
          responses: {}
          x-amazon-apigateway-integration:
            uri: "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:<region>:<account_id>:function:example-func/invocations"
            passthroughBehavior: "when_no_match"
            httpMethod: "POST"
            type: "aws_proxy"
  Description: "an api"

And now you can use variables to generalize it.

Other Automation Methods

The examples in this article are basic and simple and was done using cloudformation but this concept could be generalized where you could use terraform or other automation tool the same way and it's mostly helpful when you have a complicated resource that the console UI could help guide you to create if you're using it for the first time.





CloudFormation security validation

Integrate cfn-nag to validate all CloudFormation files

Nov 15th 2019

Joao Carvalho

DevOps on AWS like a Rockstar

DevOps Automation on AWS

Jun 6th 2019

Alan Kiš

Custom Resource with CloudFormation

How to invoke Custom Resources each time when your CloudFormation Stack is being updated.

Jun 1st 2019

David Krohn