Short summary on a previous steps - we have created network infrastructure in VPC, created necessary IAM roles on AWS, and obtained access token for our docker hub to access private images. We have also created variables file where we have stored our findings.
At that moment we need to introduce few facts: environment name (prod, staging, qa, etc), target region (shared with our network play that creates VPC), pem key to access instances if any, and naming pattern for the cluster name (prod-cluster, staging-cluster) and so on.
env: test # SET IT EXTERNALLY, desired environment name.
aws_region: "us-east-1"
ec2_key: "{{ec2_region_key | default('DEVOPS_US_EAST_1')}}" # pem key used to login to the instances
ecs_cluster_name: "{{env}}-cluster" # desired cluster name pattern, ALLOWED TO BE SET EXTERNALLY
Cluster creation is qute simple
- name: ECS | Cluster creation
shell: "aws ecs create-cluster --cluster-name {{ecs_cluster_name}} --region {{aws_region}}"
register: cluster_output
tags:
- create
- name: FACT cluster - cluster_output.stdout|from_json
set_fact:
cluster: "{{ cluster_output.stdout|from_json }}"
tags:
- create
# "cluster.cluster": {
# "activeServicesCount": 0,
# "clusterArn": "arn:aws:ecs:eu-west-1:blabla:cluster/test_cluster",
# "clusterName": "test_cluster",
# "pendingTasksCount": 0,
# "registeredContainerInstancesCount": 0,
# "runningTasksCount": 0,
# "status": "ACTIVE"
# }
- debug: var=cluster.cluster
tags:
- create
At that moment we already have cluster created.
Now we need to create several instances to operate.
Those instances are created using following pattern - we start specific AMI, with desired instance type (t2.micro is free tier), that instance should be granted with ecsInstanceRole. Re:assign_public_ip_address - if you recall our network scheme, our api servers are supposed to be in a private network. Thus properly, that needs to be set to False, + you need bastion host configured. For purposes of the demo, or quick troubleshouting, you can still make them open to the world in a quite hakish workd.
ec2_basic: {
image_id: "{{ec2_basic_image_id | default('ami-ba722dc0')}}",
instance_type: "t2.micro",
instance_profile_name: "ecsInstanceRole",
assign_public_ip_address: "yes",
vpc_subnet_id: "{{aws_vpc_pubsubnet1}}",
group_id: "{{aws_sg_api}}" # security
}
Let’s put them up, and wait for the initialization
- ec2:
count: "{{instance_count | default(1)}}"
instance_type: "{{item.instance_type}}"
instance_profile_name: "{{item.instance_profile_name}}"
instance_tags:
Name: "{{ecs_cluster_name}}"
image: "{{item.image_id}}"
group_id: "{{item.group_id}}"
region: "{{aws_region}}"
key_name: "{{ec2_key}}"
user_data: "{{ lookup('file', instance_config_path ) }}"
wait: yes
wait_timeout: 500
vpc_subnet_id: "{{item.vpc_subnet_id}}"
assign_public_ip: "{{item.assign_public_ip_address}}"
with_items:
- "{{ec2}}"
register: box_instances_raw #box_instances
- debug: var="box_instances_raw"
- FACT: cluster_instances - box_instances_raw.results[0].instances
set_fact:
cluster_instances: "{{box_instances_raw.results[0].instances}}"
- add_host: name={{ item.public_ip }} groups={{group_cluster_instances}} =""
with_items: "{{cluster_instances}}"
# NOTE - IF INSTANCES ARE PRIVATE, CODE BELOW SHOULD BE COMMENTED OUT
- name: Wait for the cluster_instances ssh availability
wait_for: host={{item.public_ip}} port=22 delay=40 timeout=320 state=started
with_items: "{{cluster_instances}}"
Note: if you have some external components, to be consumed by your application, like RDS database (Postgres, MySQL, etc), some caches like ElastiCache (Memcached, Redis) - it is good idea to create those
Hurray, we now have cluster and corresponding ec2 instances workers. Those setup is ready for our first provision.
BUT, before we do that let’s also draft steps to eliminate all the items we have created