Our team is focused on containerization using Docker and orchestration using Kubernetes. As we are a Cloud team, we wanted to explore different ways of deploying containers in the cloud.
Making research about deploying Docker containers on ECS we found out about Docker ECS Integration. As Docker says ‘The Docker ECS Integration enables developers to use native Docker commands to run applications in Amazon EC2 Container Service (ECS) when building cloud-native applications.’ they really mean it. ECS integration uses the same docker-compose.yml file for deployment in the cloud.
Why is this excellent?! If you want to deploy a solution to AWS ECS, for example using Code Build and Code Deploy, you need to set up these AWS services with additional permissions and configuration that takes time and slows down the process. As a developer, you need to configure ECS clusters, tasks, VPC, etc. And besides you already have a docker-compose.yml file, you’ll need to create an additional one buildspec.yml for CodeBuild setup that has a similar effect as docker-compose but still differs in structure. So, you end up with additional files and configurations.
Docker ECS integration is an integration solution built from AWS and Docker teams, that provides quick deployment directly into AWS ECS (Elastic Container Service) using the Docker CLI and same docker-compose.yml.
Install Docker Desktop Edge
Docker ECS Integration is currently a beta release. For this blog post, I’m using Docker Desktop Edge v19.03.13-beta2. Yes, Docker Desktop Edge is different from Docker Desktop. Before you begin, install Docker Desktop Edge and make sure that under Command Line -> Enable experimental features are enabled.
Check your installation by running the command:
>docker ecs version.
Docker ECS Setup
The next step is to set up the AWS profile and region by running:
>docker ecs setup
In the setup, you define context, region, and AWS credentials. I’m using temporary credentials, so only the first time I enter the profile name with temporary credentials. After they expire, I just re-generate new ones under the same profile name on my local /.aws/credentials and no additional entries are required.
If you are going to use an IAM user, make sure that you know the AWS Access Key ID and AWS Secret Access Key with the required permissions to support creating and managing several services in AWS:
https://github.com/docker/ecs-plugin/blob/master/docs/requirements.md
- ec2:DescribeSubnets
- ec2:DescribeVpcs
- iam:CreateServiceLinkedRole
- iam:AttachRolePolicy
- cloudformation:*
- ecs:*
- logs:*
- servicediscovery:*
- elasticloadbalancing:*
Run Compose Applications
Now, that you have integration with AWS, let’s check several things before run>docker ecs compose up:
- check docker-compose.yml file and make sure that you are not using local images. Images can be places on DockerHub, ECR, but make sure that the repo is accessible,
- docker ecs compose up will run ECS services using the AWS Fargate serverless compute engine and create ECS Cluster, permissions, and VPC for you.
Run the command in the folder where your docker-compose.yml file is located. After a minute or two in my case, the output is this:
The definitions for custom Laravel app, MySQL, and nginx are in the docker-compose.yml file. Laravel app image is committed in Docker Hub. As you can see from the output, network, log group, service discovery, and Load Balancer are completely managed by AWS. You can see real-time logs by running >docker ecs compose logs in a different CLI window from the same folder.
In the background, the Docker ECS integration plugin creates a CloudFormation file, defining the required resources for each service from the docker-compose file. It creates an ECS cluster and for each service, it registers a task definition and runs an instance of the service in the ECS Cluster, using AWS Fargate.
You can see CF stack from AWS console or by running >docker ecs compose convert and the CF file will be written out to your console.
AWS ECS Dashboard and Access Application
Let’s see what is created on AWS. Navigate to ECS. There are a cluster, tasks, and running services from definitions that are defined in docker-compose.yml file.
Try to access your application. Under tab Tasks, click on Task ID number:
Paste the Public IP address in the Browser. For Laravel you’ll see a welcome screen:
For AWS service definitions, a complete CloudFormation stack is created. CloudFormation template is created by Docker for deploying applications on Amazon ECS as the description says.
If you explore the CloudFormation stack, you’ll find definitions for:
1. Define and create ECS cluster with Task definitions and services:
- AWS:ECS::Cluster,
- AWS::ECS::Service
- AWS::ECS::TaskDefinition
2. Discover services using AWS Cloud Map API for ECS services:
- AWS::ServiceDiscovery::Service
- AWS::ServiceDiscovery::PrivateDnsNamespace
3. Add Load balancers in front of App and database:
- AWS::ElasticLoadBalancingV2::Listener
- AWS::ElasticLoadBalancingV2::TargetGroup
- AWS::ElasticLoadBalancingV2::LoadBalancer
4. Create CloudWatch log group:
- AWS::Logs::LogGroup
5. Define Roles for task execution per task definition:
- AWS::IAM::Role
6. Add security groups for services:
- AWS::EC2::SecurityGroup
- AWS::EC2::SecurityGroupIngress
Clean up
Using >docker ecs compose down you can easily clean up the ECS cluster and everything that runs inside it. This command initiates CF stack delete and clears up created resources.
Conclusion
If you’ve already had docker-compose files locally, and wonder how to deploy an application on AWS Cloud, Docker ECS Integration is here for quick and easy deployment. Keep in mind that the plugin is still in beta version, so expect upgrades. As Docker says: Today this functionality is being made available as a beta UX using docker ecs to drive commands. Later this year when the functionality becomes generally available this will become part of our new Docker Contexts and will allow you to just run docker run and docker compose.
