Shortcuts for AWS Infrastructure Automation

 
 
 

AWS infrastructure automation is time consuming and at certain times the tooling documenation like cloudformation and terraform does not have clear documentation of the attributes. But for certain AWS services like the ones I'll be discussing in this post you can use the AWS Console UI to guide to create a version of the object and then grab the attributes from the object through the cli and convert it to cloudformation or terraform then you can use variable references instead of hard coding to generalize the object.

AWS Codepipeline Project

I'll start by assuming a codepipeline project was created in the console and was named example-pipeline, and you have aws cli v2 installed and creds configured.

  • First we get the pipeline information and store it.
aws2 codepipeline get-pipeline \
  --name frontend-pipeline-prod-from-production \
  --output yaml > example_pipeline.yaml

And this is a sample content of the example_pipeline.yaml file:

metadata:
  created: '2019-11-22T16:32:48.387000+01:00'
  pipelineArn: arn:aws:codepipeline:<region>:<account_id>:example-pipeline
  updated: '2019-11-22T16:42:02.548000+01:00'
pipeline:
  artifactStore:
    location: example-artifact-bucket
    type: S3
  name: example-pipeline
  roleArn: arn:aws:iam::<account_id>:role/codepipeline-role
  stages:
  - actions:
    - actionTypeId:
        category: Source
        owner: ThirdParty
        provider: GitHub
        version: '1'
      configuration:
        Branch: master
        OAuthToken: '****'
        Owner: foo
        PollForSourceChanges: 'true'
        Repo: example-repo
      inputArtifacts: []
      name: Source
      outputArtifacts:
      - name: code
      runOrder: 1
    name: Source
  - actions:
    - actionTypeId:
        category: Build
        owner: AWS
        provider: CodeBuild
        version: '1'
      configuration:
        EnvironmentVariables: "[{\n\t\"name\": \"ENVIRONMENT\",\n\t\"value\": \"prod\",\n\t\"type\":
          \"PLAINTEXT\"\n}]\n"
        ProjectName: sample-build
      inputArtifacts:
      - name: code
      name: Build
      outputArtifacts: []
      runOrder: 2
    name: deploy-step
  version: 3
  • We can convert this to cloudformation like this since all the attributes are just there as in this sample snippet:
Type: AWS::CodePipeline::Pipeline
Properties:
  ArtifactStore:
    Location: example-artifact-bucket
    Type: S3
  Name: example-pipeline
  RoleArn: arn:aws:iam::<account_id>:role/codepipeline-role
  Stages:
  - Actions:
    - ActionTypeId:
        Category: Source
        Owner: ThirdParty
        Provider: GitHub
        Version: '1'
      Configuration:
        Branch: master
        OAuthToken: '****'
        Owner: foo
        PollForSourceChanges: 'true'
        Repo: example-repo
      InputArtifacts: []
      Name: Source
      OutputArtifacts:
      - Name: code
      RunOrder: 1
    Name: Source
  - Actions:
    - ActionTypeId:
        Category: Build
        Owner: AWS
        Provider: CodeBuild
        Version: '1'
      Configuration:
        EnvironmentVariables: "[{\n\t\"name\": \"ENVIRONMENT\",\n\t\"value\": \"prod\",\n\t\"type\":
          \"PLAINTEXT\"\n}]\n"
        ProjectName: sample-build
      InputArtifacts:
      - Name: code
      Name: Build
      OutputArtifacts: []
      RunOrder: 2
    Name: deploy-step

As could be seen from the conversion all you need to do is change the attribute names from camel case to pascal to fit them as a cloudformation resource, the next step would be to change certain values to variables to generalize the template but that's case specific and could be left for the reader.

AWS ApiGateway

I'll start by assuming an api gateway is created an deployed using the console and has this id "fffffffffd" and this stage name "test", and you have aws cli v2 installed and creds configured.

  • This one is simpler, all you need to do is grab the extended swagger and drop it inside the cloudformation template and create variables/generalize.
aws2 apigateway get-export \
 --parameters extensions='apigateway' \
 --accepts application/yaml \
 --rest-api-id fffffffffd \
 --stage-name test \
 --export-type swagger \
 ./the_swagger.yaml

And this is a sample swagger output:

---
swagger: "2.0"
info:
  version: "2020-01-09T06:49:46Z"
  title: "example-api"
host: "example.com"
schemes:
- "https"
paths:
  /the-example:
    post:
      responses: {}
      x-amazon-apigateway-integration:
        uri: "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:<region>:<account_id>:function:example-func/invocations"
        passthroughBehavior: "when_no_match"
        httpMethod: "POST"
        type: "aws_proxy"
  • Now just add this to the cloudformation template, for example:
Type: AWS::ApiGatewayV2::Api
Properties:
  BasePath: "/foo"
  Body:
    swagger: "2.0"
    info:
      version: "2020-01-09T06:49:46Z"
      title: "example-api"
    host: "example.com"
    schemes:
    - "https"
    paths:
      /the-example:
        post:
          responses: {}
          x-amazon-apigateway-integration:
            uri: "arn:aws:apigateway:eu-west-1:lambda:path/2015-03-31/functions/arn:aws:lambda:<region>:<account_id>:function:example-func/invocations"
            passthroughBehavior: "when_no_match"
            httpMethod: "POST"
            type: "aws_proxy"
  Description: "an api"

And now you can use variables to generalize it.

Other Automation Methods

The examples in this article are basic and simple and was done using cloudformation but this concept could be generalized where you could use terraform or other automation tool the same way and it's mostly helpful when you have a complicated resource that the console UI could help guide you to create if you're using it for the first time.




      

CloudFormation Best Practices

Recommendations that can help you to use CloudFormation more effectively and securely throughout its entire workflow.

 

Oct 20th 2020

 

David Krohn

Share your ACM Private CA cross-account

How to automate AWS RAM to share your ACM Private CA cross-account

 

Sep 3rd 2020

 

David Krohn

App Mesh and ECS with shared Meshes

Step by Step Tutorial on how to use App Mesh on ECS with cross account ressource sharing (shared meshes)

 

Jul 25th 2020

 

Yassine Berrada Touati