Provisioning solution to amazon ecs, part 2 - preparing needed dependencies¶

In order to be ready to create our first cluster, we need to prepare some parameters in advance. Those are:

1. If you are going to use ssl , we need to note ssl certificate id (arn)

project_ssl_certificate_arn: “arn:aws:acm:us-east-1:blablabla”

2. We need to chose operation system on instances which would power our cluster.

At a moment, two most often used ones - are Amazon optimized image with agent, https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html and CoreOS https://coreos.com/os/docs/latest/booting-on-ec2.html ;

Usually I start with Amazon optimized, while project is under development, as it is easier to troubleshoot, and switch to CoreOS once project reaches production quality.

Write down current AMI image id for your region at a moment.

3. We need to pre-create to roles in our account - ecsServiceRole and ecsInstanceRole

ecsServiceRole_arn: “arn:aws:iam::blabla:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS” ecsInstanceRole_arn: “arn:aws:iam::blabla:instance-profile/ecsInstanceRole”

Note: depending on exact architecture of the application, you might provide additional set, check out https://docs.aws.amazon.com/AmazonECS/latest/developerguide/IAM_policies.html

As service improves, requirements change from time to time, make sure to consult documentation. At a moment of article writing, ecsServiceRole policy for purposes of the demo can be described as

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ec2:AttachNetworkInterface",
                "ec2:CreateNetworkInterface",
                "ec2:CreateNetworkInterfacePermission",
                "ec2:DeleteNetworkInterface",
                "ec2:DeleteNetworkInterfacePermission",
                "ec2:Describe*",
                "ec2:DetachNetworkInterface",
                "elasticloadbalancing:DeregisterInstancesFromLoadBalancer",
                "elasticloadbalancing:DeregisterTargets",
                "elasticloadbalancing:Describe*",
                "elasticloadbalancing:RegisterInstancesWithLoadBalancer",
                "elasticloadbalancing:RegisterTargets"
            ],
            "Resource": "*"
        }
    ]
}

and ecsInstanceRole role policy (referred also as AmazonEC2ContainerServiceforEC2Role ) can be described as

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "ecs:CreateCluster",
                "ecs:DeregisterContainerInstance",
                "ecs:DiscoverPollEndpoint",
                "ecs:Poll",
                "ecs:RegisterContainerInstance",
                "ecs:StartTelemetrySession",
                "ecs:UpdateContainerInstancesState",
                "ecs:Submit*",
                "ecr:GetAuthorizationToken",
                "ecr:BatchCheckLayerAvailability",
                "ecr:GetDownloadUrlForLayer",
                "ecr:BatchGetImage",
                "logs:CreateLogStream",
                "logs:PutLogEvents"
            ],
            "Resource": "*"
        }
    ]
}

Note: for any concrete case you should follow rule of less permissions given. I.e. your roles should cover only rights that are really needed for your application

4. Docker registry credentials

You have also possibility since approx end of 2016 to host your docker images on a aws provided registry, or, you can use docker registry of your choice, for example, docker hub. Upon login token can be obtained from .docker/config.json

Example:

mkdir -p .docker || echo ".docker directory exists"
export DOCKER_CONFIG=`pwd`/.docker
docker login --username=dockerhub@login.dev --password='YOURSECUREPASSWORD'
cat .docker/config.json # prove that the file contains credentials

5. And of course infastructure variables in aws-us-east-1-VPC-vars.yml from the network step , see first part of the article, if you missed

---
# {{ ansible_managed }}
# This file was generated by ansible via __network_create.yml
aws_vpc_id: {{ lookup('aws_vpc_id_from_name', aws_region, readable_env_name + '-vpc-' + aws_region) }}
aws_vpc_privsubnet1: {{aws_vpc_privsubnet1 | default(aws_vpc_privsubnet1_runtime)}}
aws_vpc_privsubnet2: {{aws_vpc_privsubnet2 | default(aws_vpc_privsubnet2_runtime)}}
aws_vpc_pubsubnet1: {{aws_vpc_pubsubnet1 | default(aws_vpc_pubsubnet1_runtime)}}
aws_vpc_pubsubnet2: {{aws_vpc_pubsubnet2 | default(aws_vpc_pubsubnet2_runtime)}}
aws_sg_pub: {{aws_sg_pub | default(lookup('aws_secgroup_ids_from_names', aws_region, [readable_env_name + '-public-ELB']))}}
aws_sg_api: {{aws_sg_api | default(lookup('aws_secgroup_ids_from_names', aws_region, [readable_env_name + '-private-CLUSTER']))}}
aws_sg_priv: {{aws_sg_priv | default(lookup('aws_secgroup_ids_from_names', aws_region, [readable_env_name + '-private-DATALAYER']))}}

Let’s summarize, what we have in hypothetical cluster_defaults.yml:

project_ssl_certificate_arn: "arn:aws:acm:us-east-1:blablabla"

ecsServiceRole_arn: "arn:aws:iam::blabla:role/aws-service-role/ecs.amazonaws.com/AWSServiceRoleForECS"
ecsInstanceRole_arn: "arn:aws:iam::blabla:instance-profile/ecsInstanceRole"


# https://docs.aws.amazon.com/AmazonECS/latest/developerguide/ecs-optimized_AMI.html
ec2_basic: {
 image_id: "{{ec2_basic_image_id | default('ami-ba722dc0')}}",
 instance_type: "t2.micro",
 instance_profile_name: "ecsInstanceRole",
 assign_public_ip_address: "yes",
 vpc_subnet_id: "{{aws_vpc_pubsubnet1}}",
 group_id: "{{aws_sg_api}}" # security
}

# https://coreos.com/os/docs/latest/booting-on-ec2.html
ec2_coreos: {
 image_id: "{{ec2_coreos_image_id | default('ami-e582d29f')}}",
 instance_type: "t2.micro",
 instance_profile_name: "ecsInstanceRole",
 assign_public_ip_address: "yes",
 vpc_subnet_id: "{{aws_vpc_pubsubnet1}}",
 group_id: "{{aws_sg_api}}" # security
}

ec2: "{{ec2_basic}}"  # selected type

ecs_engine_auth_data_token: "CENSORED_TOKEN"  # todo: SET IT FROM SECURE VARS , cat ~/.docker/config.json
ecs_engine_auth_data_email: "CENSORED_LOGIN"  # todo:  SET IT FROM SECURE VARS

That’s all, we are ready for the next step