.Blue-Green deployments using AWS Fargate and AWS CodeDeploy

The Modern application development lifecycle is based on deploying new features and bug fixes up to multiple times a day. To accomplish this, there needs to be a way of providing stable and predictable deployments. This means the end user should not be shown the fixed or updated version of the application, unless the changes have been verified to be working as intended. Thankfully, there are many different deployment strategies, of which one stands in the limelight: blue/green deployments.

Blue/green Deployments

At a high level, blue/green deployments work like this: there is a group of instances (e.g., containers) running one version of the application.This is the blue group. When there is an application update that needs to be published, we do not want all of the users currently using the application immediately dropping their sessions. Instead, a new group of instances is spun up. This group is the green group of instances. Before making the switch between the blue and the green group, we can run tests to make sure that the application version running in the green group behaves as expected. In a situation where it is a web application, a simple HTTP GET request to an endpoint could give enough information to proceed. If the tests fail, the application will keep running on the blue group of instances and no switch will be made (phew, we do not want any angry users!). If the tests are successful, all (or some of) the traffic is switched from the blue to the green group and the blue group instances are (gradually) terminated. Even though this explanation is a simplified version of how blue/green works in practice, it should depict what the goal of using this deployment strategy is: minimizing the downtime. If done correctly, there should be no downtime, i.e., we will achieve what is called zero downtime deployment. There are many great articles online covering the blue/green deployment topic, so we will not dive any further into it right now.

The aim of this blog post is to show how blue/green deployments can be performed using AWS Fargate and CodeDeploy with the added benefit of everything being serverless.

Prerequisites

The first step is creating everything that is required: an ECR repository, an ECS cluster, a task definition, an ALB, CodeDeploy application and a corresponding deployment group. Since we are using Fargate (i.e., containers) we will need a Docker image for our application as well. If your Docker image is in a different container registry (e.g. Docker Hub), you can just reference it in your task definition file instead of creating an ECR repository. The web application that was used for the initial testing was a simple Python web application. Similar examples of the application can be found in FastAPI documentation.

Before we proceed, one thing to note here is that there is already a built-in rollback mechanism in ECS called deployment circuit breaker. We have found out that deployment circuit breaker and blue/green with CodeDeploy cannot be used together since they are using completely different (and incompatible) deployment controllers, but they do provide a similar result. A code snippet for setting up the CodeDeploy controller in the ECS service (using Terraform):

 deployment_controller {
    type = "CODE_DEPLOY"
 }

A somewhat simplified architecture diagram is shown below.

When CodeDeploy is used you get an additional feature which enables defining of hooks during different stages of deployment. To anyone familiar with the CodeDeploy hooks, this will make sense, however there is one caveat: there is some difference between ECS and EC2 instances as far as the hooks that are available. This difference exists because ECS (in this case, Fargate) is used to deploy applications in containers. In this blog post, we will focus on using only one of these hooks, BeforeAllowTraffic. An example appspec.yaml file for CodeDeploy:

version: 0.0
Resources:
  - TargetService:
      Type: AWS::ECS::Service
      Properties:
        TaskDefinition: "<task_definition_arn>"
        LoadBalancerInfo:
          ContainerName: "<container_name>"
          ContainerPort: <app_port>
Hooks:
  - BeforeAllowTraffic: "<lambda_hook_function>"

Since we want to test if our web application is reachable or not, a BeforeAllowTraffic hook should be a valid test. In other words, if the tests fail no traffic will be allowed to the green group. To make this work, we need to allow CodeDeploy to perform the tests on our behalf. This will require a Lambda function that CodeDeploy will trigger when it reaches the BeforeAllowTraffic lifecycle event. The Lambda function will perform a simple curl on the web application endpoint we want to test and return the result. If the result is different than HTTP 200 OK, the green group will be decommissioned and the blue group will continue to be operational.

If on the other hand the response is one that we expect, the blue group will be decommissioned and the green group will take over serving any traffic coming from end users.

A very important thing to note is that if you are using an Infrastructure-as-Code (IaC) tool such as Terraform, the CodeDeployment deployment cannot be triggered (i.e., there is no Terraform resource for that). However, if you are doing this as a part of a CI/CD pipeline (as you should!), an AWS CLI command can be used to start a new deployment. The CLI parameters can be provided one by one or by using a deployment JSON file with all the parameters provided. An example JSON file for the deployment:

{
    "applicationName": "<aws_codedeploy_app_name>",
    "deploymentGroupName": "<aws_codedeploy_group_name>",
    "revision": {
        "revisionType": "S3",
        "s3Location": {
            "bucket": "<appspec_s3_bucket>",
            "key": "appspec.yaml",
            "bundleType": "YAML"
        }
    }
}

Bearing in mind the command is run from the same directory where deployment.json file is, the complete command that would be used is as follows:

aws deploy create-deployment --cli-input-json file://deployment.json --region <your CodeDeploy region>

If you are using GitHub actions, there is even an action provided by AWS which can be used.

Conclusion

The Blue/green deployment strategy is a great way to test changes made to your application prior to allowing it to be accessed, avoiding unnecessary issues. Additionally the integration that exists between AWS Fargate and CodeDeploy means configuration takes minimal effort. The setup of the solution outlined in this blog post is based on serverless technologies, therefore providing a cost-effective solution.