Get Started¶
In this document, we expect to start ec2 instances, deploy software stacks with a dynamic inventory like a text inventory file and terminate instances when its completed. This can be applied to other cloud platforms such as Microsoft Azure or OpenStack.
Background¶
Typical ansible playbook runs with a static text inventory file which consists of a set of IP addresses or host names accessible via SSH because it assume that there are machines ready to install software and configure settings through Ansible roles, plays. When it comes to cloud resources, those IPs or host names are not permanent because new instances are started with different network access information. Ansible Dynamic Inventory allows to retrieve instance information directly from EC2 instead of reading a text file.
In addition, Ansible ec2 cloud module supports creating/terminating ec2
instances as Ansible tasks, therefore booting and terminating ec2 instances are
automated by Ansible plays, in this case, boot.yml and terminate.yml
files.
AWS Credentials¶
AWS_ACCESS_KEY and AWS_SECRET_ACCESS_KEY are required to communicate
with AWS under your account. Ansible provides several options to import
credentials like ec2.ini file, boto profile (e.g. ~/.aws/credentials) and
environment variables but we will use environment variables with
ansible-vault to secure the information.
Be ready with your
AWS_ACCESS_KEYandAWS_SECRET_ACCESS_KEYStore the information in a file like:
$ nano cred export AWS_ACCESS_KEY=1234567890 export AWS_SECRET_ACCESS_KEY=1234567890
(Replace 1234567890 with a real value)
Encrypt the file with
ansible-vaultwith your password for this file:$ ansible-vault encrypt cred New Vault password: Confirm New Vault password:
The encrypted file looks like:
$ cat cred
$ANSIBLE_VAULT;1.1;AES256
36383835346339373263616238633662613265653766646236616538326130616666373864323537
6265613637646530396663393236636534313138623536300a343638643134656330646661653230
32363234646631323034336338376337323836616532396639656234396235303531306133323663
3838383962386338320a623932313934356563303964303831323732323363343364646235633261
61396361366131363262623739376535393062663361303966333533396664663832616234306234
34623461316530643063313338666635396463336361643962633536666264613665343135303163
613937373831323931613039653438363431
Enable your EC2 credentials in a shell by:
$ $(ansible-vault view cred) Vault password: $
You can confirm whether it’s loaded or not by:
$ env|grep AWS
AWS_SECRET_ACCESS_KEY=1234567890
AWS_ACCESS_KEY=1234567890
Dynamic Inventory¶
It is not required to have a text inventory file in your Ansible home directory
as long as ec2.py script provides dynamic inventory.
Download and run
ec2.pyAWS External Inventory Script:$ curl -L https://raw.github.com/ansible/ansible/devel/contrib/inventory/ec2.py > ec2.py % Total % Received % Xferd Average Speed Time Time Time Current Dload Upload Total Spent Left Speed 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0 100 63776 100 63776 0 0 135k 0 --:--:-- --:--:-- --:--:-- 317k
Make it executable by:
$ chmod a+x ./ec2.py
Check running EC2 instances, (wait a few seconds):
$ ./ec2.py { "_meta": { "hostvars": {} } }
There is no running instances now but if you have some, the JSON return dataset looks like:
{ "_meta": { "hostvars": { "54.85.130.171": { "ansible_ssh_host": "54.85.130.171", "ec2__in_monitoring_element": false, "ec2_ami_launch_index": "0", "ec2_architecture": "x86_64", "ec2_client_token": "", "ec2_dns_name": "", "ec2_ebs_optimized": false, "ec2_eventsSet": "", "ec2_group_name": "", "ec2_hypervisor": "xen", "ec2_id": "i-1a2c68e2", "ec2_image_id": "ami-2d39803a", "ec2_instance_profile": "", "ec2_instance_type": "t2.micro", "ec2_ip_address": "54.85.130.171", "ec2_item": "", "ec2_kernel": "", "ec2_key_name": "albert", "ec2_launch_time": "2016-09-15T17:18:36.000Z", "ec2_monitored": false, "ec2_monitoring": "", "ec2_monitoring_state": "disabled", "ec2_persistent": false, "ec2_placement": "us-east-1a", "ec2_platform": "", "ec2_previous_state": "", "ec2_previous_state_code": 0, "ec2_private_dns_name": "ip-172-30-3-104.ec2.internal", "ec2_private_ip_address": "172.30.3.104", "ec2_public_dns_name": "", "ec2_ramdisk": "", "ec2_reason": "", "ec2_region": "us-east-1", "ec2_requester_id": "", "ec2_root_device_name": "/dev/sda1", "ec2_root_device_type": "ebs", "ec2_security_group_ids": "sg-69ebc60c,sg-115b376b", "ec2_security_group_names": "default,abds_secgroup", "ec2_sourceDestCheck": "true", "ec2_spot_instance_request_id": "", "ec2_state": "running", "ec2_state_code": 16, "ec2_state_reason": "", "ec2_subnet_id": "subnet-719a774d", "ec2_tag_datanodes": "True", "ec2_tag_frontendnodes": "False", "ec2_tag_hadoopnodes": "True", "ec2_tag_historyservernodes": "False", "ec2_tag_journalnodes": "True", "ec2_tag_namenodes": "True", "ec2_tag_resourcemanagernodes": "True", "ec2_tag_zookeepernodes": "True", "ec2_virtualization_type": "hvm", "ec2_vpc_id": "vpc-e6c17c83" } } }, "ami_2d39803a": [ "54.85.130.171" ], "ec2": [ "54.85.130.171" ], "i-1a2c68e2": [ "54.85.130.171" ], "key_albert": [ "54.85.130.171" ], "security_group_abds_secgroup": [ "54.85.130.171" ], "security_group_default": [ "54.85.130.171" ], "tag_datanodes_True": [ "54.85.130.171" ], "tag_frontendnodes_False": [ "54.85.130.171" ], "tag_hadoopnodes_True": [ "54.85.130.171" ], "tag_historyservernodes_False": [ "54.85.130.171" ], "tag_journalnodes_True": [ "54.85.130.171" ], "tag_namenodes_True": [ "54.85.130.171" ], "tag_resourcemanagernodes_True": [ "54.85.130.171" ], "tag_zookeepernodes_True": [ "54.85.130.171" ], "type_t2_micro": [ "54.85.130.171" ], "us-east-1": [ "54.85.130.171" ], "us-east-1a": [ "54.85.130.171" ], "vpc_id_vpc_e6c17c83": [ "54.85.130.171" ] }
Switch inventory argument option in your
ansible-playbookcommand like:$ ansible-playbook -i ec2.py
For more information, see Ansible Dynamic Inventory here
Launching EC2 Instances via Ansible ec2 module¶
Ansible provide cloud modules to manage instances and we use ec2 module for
Amazon EC2 cloud to launch or terminate instances. Other clouds i.e. Azure,
Digital Ocean, Docker, Google Compute Engine (GCE) or OpenStack are also
supported in the cloud modules. For more information see documentation here and source code
from ansible-modules-core here
The following ec2 task starts a single EC2 instance with Ubuntu 14.04 image:
- ec2:
key_name: albert # SSH registered key name
instance_type: t2.micro # EC2 Instance size
image: ami-2d39803a # Ubuntu 14.04 image
group: default # security group
region: us-east-1 # EC2 Region
This five lines in ec2 task launches a new EC2 instance on us-east-1 region.
Find out more details here
EC2 Tags for Inventory Groups¶
instance_tags is used to create inventory groups therefore Ansible plays
run on particular hosts. For example, deploying hadoop cluster requires
several inventory groups such as namenodes, datanodes or zookeepernodes.
instance_tags is useful to put a label on ec2 instances to distinguish
virtual servers between different inventory groups. The following example shows
that three ec2 instances will be created with the tags for namenodes, datanodes
and zookeepernodes like:
- ec2:
key_name: "{{ keypair }}"
instance_type: "{{ instance_size }}"
image: "{{ image }}"
region: "{{ region }}"
exact_count: 3
count_tag: three-nodes
group: [ 'default', "{{ security_group }}" ]
instance_tags:
namenodes: True
datanodes: True
zookeepernodes: True
historyservernodes: False
frontendnodes: False
The key:value tags are used here and the AWS EC2 returns JSON data like:
{u'ansible_all_ipv4_addresses': [u'172.30.3.139'],
u'ansible_all_ipv6_addresses': [u'fe80::4b9:ffff:fe81:65f'],
u'ansible_architecture': u'x86_64',
u'ansible_bios_date': u'05/12/2016',
u'ansible_bios_version': u'4.2.amazon',
'ansible_check_mode': False,
u'ansible_cmdline': {u'BOOT_IMAGE': u'/boot/vmlinuz-3.13.0-91-generic',
u'console': u'ttyS0',
u'ro': True,
u'root': u'UUID=73362e04-90b8-4c53-b878-2097820e0b34'},
u'ansible_date_time': {u'date': u'2016-09-16',
u'day': u'16',
u'epoch': u'1473991427',
u'hour': u'02',
u'iso8601': u'2016-09-16T02:03:47Z',
u'iso8601_basic': u'20160916T020347397930',
u'iso8601_basic_short': u'20160916T020347',
u'iso8601_micro': u'2016-09-16T02:03:47.398008Z',
u'minute': u'03',
u'month': u'09',
u'second': u'47',
u'time': u'02:03:47',
u'tz': u'UTC',
u'tz_offset': u'+0000',
u'weekday': u'Friday',
u'weekday_number': u'5',
u'weeknumber': u'37',
u'year': u'2016'},
u'ansible_default_ipv4': {u'address': u'172.30.3.139',
u'alias': u'eth0',
u'broadcast': u'172.30.3.255',
u'gateway': u'172.30.3.1',
u'interface': u'eth0',
u'macaddress': u'06:b9:ff:81:06:5f',
u'mtu': 9001,
u'netmask': u'255.255.255.0',
u'network': u'172.30.3.0',
u'type': u'ether'},
u'ansible_default_ipv6': {},
u'ansible_devices': {u'xvda': {u'holders': [],
u'host': u'',
u'model': None,
u'partitions': {u'xvda1': {u'sectors': u'16755795',
u'sectorsize': 512,
u'size': u'7.99 GB',
u'start': u'16065'}},
u'removable': u'0',
u'rotational': u'0',
u'sas_address': None,
u'sas_device_handle': None,
u'scheduler_mode': u'deadline',
u'sectors': u'16777216',
u'sectorsize': u'512',
u'size': u'8.00 GB',
u'support_discard': u'0',
u'vendor': None}},
u'ansible_distribution': u'Ubuntu',
u'ansible_distribution_major_version': u'14',
u'ansible_distribution_release': u'trusty',
u'ansible_distribution_version': u'14.04',
u'ansible_dns': {u'nameservers': [u'172.30.0.2'],
u'search': [u'ec2.internal']},
u'ansible_domain': u'',
u'ansible_env': {u'HOME': u'/home/ubuntu',
u'LANG': u'en_US.UTF-8',
u'LC_ALL': u'en_US.UTF-8',
u'LC_MESSAGES': u'en_US.UTF-8',
u'LOGNAME': u'ubuntu',
u'MAIL': u'/var/mail/ubuntu',
u'PATH': u'/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/games:/usr/local/games',
u'PWD': u'/home/ubuntu',
u'SHELL': u'/bin/bash',
u'SHLVL': u'1',
u'SSH_CLIENT': u'156.56.102.202 38016 22',
u'SSH_CONNECTION': u'156.56.102.202 38016 172.30.3.139 22',
u'SSH_TTY': u'/dev/pts/0',
u'TERM': u'xterm',
u'USER': u'ubuntu',
u'XDG_RUNTIME_DIR': u'/run/user/1000',
u'XDG_SESSION_ID': u'8',
u'_': u'/bin/sh'},
u'ansible_eth0': {u'active': True,
u'device': u'eth0',
u'ipv4': {u'address': u'172.30.3.139',
u'broadcast': u'172.30.3.255',
u'netmask': u'255.255.255.0',
u'network': u'172.30.3.0'},
u'ipv6': [{u'address': u'fe80::4b9:ffff:fe81:65f',
u'prefix': u'64',
u'scope': u'link'}],
u'macaddress': u'06:b9:ff:81:06:5f',
u'mtu': 9001,
u'pciid': u'vif-0',
u'promisc': False,
u'type': u'ether'},
u'ansible_fips': False,
u'ansible_form_factor': u'Other',
u'ansible_fqdn': u'ip-172-30-3-139',
u'ansible_gather_subset': [u'hardware', u'network', u'virtual'],
u'ansible_hostname': u'ip-172-30-3-139',
u'ansible_interfaces': [u'lo', u'eth0'],
u'ansible_kernel': u'3.13.0-91-generic',
u'ansible_lo': {u'active': True,
u'device': u'lo',
u'ipv4': {u'address': u'127.0.0.1',
u'broadcast': u'host',
u'netmask': u'255.0.0.0',
u'network': u'127.0.0.0'},
u'ipv6': [{u'address': u'::1',
u'prefix': u'128',
u'scope': u'host'}],
u'mtu': 65536,
u'promisc': False,
u'type': u'loopback'},
u'ansible_lsb': {u'codename': u'trusty',
u'description': u'Ubuntu 14.04.4 LTS',
u'id': u'Ubuntu',
u'major_release': u'14',
u'release': u'14.04'},
u'ansible_machine': u'x86_64',
u'ansible_machine_id': u'dd1bac9cb23792a632bd6ee157719517',
u'ansible_memfree_mb': 715,
u'ansible_memory_mb': {u'nocache': {u'free': 932, u'used': 60},
u'real': {u'free': 715, u'total': 992, u'used': 277},
u'swap': {u'cached': 0,
u'free': 0,
u'total': 0,
u'used': 0}},
u'ansible_memtotal_mb': 992,
u'ansible_mounts': [{u'device': u'/dev/xvda1',
u'fstype': u'ext4',
u'mount': u'/',
u'options': u'rw,discard',
u'size_available': 7029567488,
u'size_total': 8309932032,
u'uuid': u''}],
u'ansible_nodename': u'ip-172-30-3-139',
u'ansible_os_family': u'Debian',
u'ansible_pkg_mgr': u'apt',
u'ansible_processor': [u'GenuineIntel',
u'Intel(R) Xeon(R) CPU E5-2670 v2 @ 2.50GHz'],
u'ansible_processor_cores': 1,
u'ansible_processor_count': 1,
u'ansible_processor_threads_per_core': 1,
u'ansible_processor_vcpus': 1,
u'ansible_product_name': u'HVM domU',
u'ansible_product_serial': u'NA',
u'ansible_product_uuid': u'NA',
u'ansible_product_version': u'4.2.amazon',
u'ansible_python': {u'executable': u'/usr/bin/python',
u'has_sslcontext': False,
u'type': u'CPython',
u'version': {u'major': 2,
u'micro': 6,
u'minor': 7,
u'releaselevel': u'final',
u'serial': 0},
u'version_info': [2, 7, 6, u'final', 0]},
u'ansible_python_version': u'2.7.6',
u'ansible_selinux': False,
u'ansible_service_mgr': u'upstart',
u'ansible_ssh_host': u'52.23.213.103',
u'ansible_ssh_host_key_dsa_public': u'AAAAB3NzaC1kc3MAAACBAKosQQsY22MWjQv/bF7V6kewbM5Z7NIPjYgNL/j8lbUrmZv/kZSAArfs/ByKjC2gesSZUi3hZ1Lo1LmFZjcu00IUKosKnPhHF6hNoyApxHNL1FovWULIWAyAbxpsRNkBNJYQ9Ox0Sklgf1PC94GJwL4SD3lSm180YsaEIw+AC5kJAAAAFQCdQN2qo70bFXxFjSUa0mZ8ORgRSwAAAIBR+lfWXZhjvi5RCccs4xwfrsTq+ujsVsb3qRP3vE+Caxyd8HrdeTDV9jRqJEvsSmKU57L58NBiteYFHfdA0D+HOQm56ZvDEhz/FVRR4ei2UZC04TEsZ4aXfvGzt6OnVRcguiozjeWnglWhikvcdrQHxszRziiaDdubnomlfn0xMQAAAIEAgKs1pwbh9S/dE5nbSqwlkTiOR6jkcPhP8XGSvjcjXqbm+ymnjJ9W9hM7DiaFm83YjWf+3+LOLMtULf7w91xOC0HLRD5M2WWQ/RxATcLN37jhbAs3XkYqZNCtVWOGm6m/wbOiWt3R3SYS5h6Bp4oqHyTYifQsfY+KVSyb29RjZzc=',
u'ansible_ssh_host_key_ecdsa_public': u'AAAAE2VjZHNhLXNoYTItbmlzdHAyNTYAAAAIbmlzdHAyNTYAAABBBCjuapwx2AgG834ti6pGu7gbF2YJmoQ1r6wGVNQrnVHs7rZ5S+hhTSjRmZQ7d5zYgbqFX/f9WGAQgAn89dWyHPE=',
u'ansible_ssh_host_key_ed25519_public': u'AAAAC3NzaC1lZDI1NTE5AAAAIEOzako99Tn5mO/vjk+ctHc9HKmtdsdJxTqqNIoo0RI3',
u'ansible_ssh_host_key_rsa_public': u'AAAAB3NzaC1yc2EAAAADAQABAAABAQC7elCMaVXdqIuBrO+4Fx5oPggQdI7JRKv176NsoRtwsYPtxp07t4mllkH6ATsQgkj9wDHOxMk73zyEFaMZskCVNwvN1bwGIyZX/i8JXJtzAdvZZNn1cH1zCi1WD3bRx3dp+p8JU+OmVBhaElPYqm3W+BDWGC7b4YqHbqTHDNTUswgvvLuXr0Qw7fdWMNxb8Va7c3GOrMEkMWnHpMpVCu6Y8zIHdcmJWSMeBXOUKPkXXsaI6jAX8koi8pAFeX95rHbwhP2uo76ueJqWJhDPLm6FdXW+hsPIKDn4eFIhVrHNROml4Z3pAR6FmfidJEi5dvXmefLC6IQH9eDgf7Dn5fS9',
u'ansible_swapfree_mb': 0,
u'ansible_swaptotal_mb': 0,
u'ansible_system': u'Linux',
u'ansible_system_capabilities': [u''],
u'ansible_system_capabilities_enforced': u'True',
u'ansible_system_vendor': u'Xen',
u'ansible_uptime_seconds': 21477,
u'ansible_user_dir': u'/home/ubuntu',
u'ansible_user_gecos': u'Ubuntu',
u'ansible_user_gid': 1000,
u'ansible_user_id': u'ubuntu',
u'ansible_user_shell': u'/bin/bash',
u'ansible_user_uid': 1000,
u'ansible_userspace_architecture': u'x86_64',
u'ansible_userspace_bits': u'64',
'ansible_version': {'full': '2.1.1.0',
'major': 2,
'minor': 1,
'revision': 1,
'string': '2.1.1.0'},
u'ansible_virtualization_role': u'guest',
u'ansible_virtualization_type': u'xen',
u'ec2__in_monitoring_element': False,
u'ec2_ami_launch_index': u'0',
u'ec2_architecture': u'x86_64',
u'ec2_client_token': u'',
u'ec2_dns_name': u'',
u'ec2_ebs_optimized': False,
u'ec2_eventsSet': u'',
u'ec2_group_name': u'',
u'ec2_hypervisor': u'xen',
u'ec2_id': u'i-435014bb',
u'ec2_image_id': u'ami-2d39803a',
u'ec2_instance_profile': u'',
u'ec2_instance_type': u't2.micro',
u'ec2_ip_address': u'52.23.213.103',
u'ec2_item': u'',
u'ec2_kernel': u'',
u'ec2_key_name': u'albert',
u'ec2_launch_time': u'2016-09-15T20:05:34.000Z',
u'ec2_monitored': False,
u'ec2_monitoring': u'',
u'ec2_monitoring_state': u'disabled',
u'ec2_persistent': False,
u'ec2_placement': u'us-east-1a',
u'ec2_platform': u'',
u'ec2_previous_state': u'',
u'ec2_previous_state_code': 0,
u'ec2_private_dns_name': u'ip-172-30-3-139.ec2.internal',
u'ec2_private_ip_address': u'172.30.3.139',
u'ec2_public_dns_name': u'',
u'ec2_ramdisk': u'',
u'ec2_reason': u'',
u'ec2_region': u'us-east-1',
u'ec2_requester_id': u'',
u'ec2_root_device_name': u'/dev/sda1',
u'ec2_root_device_type': u'ebs',
u'ec2_security_group_ids': u'sg-69ebc60c,sg-115b376b',
u'ec2_security_group_names': u'default,abds_secgroup',
u'ec2_sourceDestCheck': u'true',
u'ec2_spot_instance_request_id': u'',
u'ec2_state': u'running',
u'ec2_state_code': 16,
u'ec2_state_reason': u'',
u'ec2_subnet_id': u'subnet-719a774d',
u'ec2_tag_datanodes': u'True',
u'ec2_tag_frontendnodes': u'True',
u'ec2_tag_hadoopnodes': u'True',
u'ec2_tag_historyservernodes': u'True',
u'ec2_tag_journalnodes': u'True',
u'ec2_tag_namenodes': u'True',
u'ec2_tag_resourcemanagernodes': u'True',
u'ec2_tag_zookeepernodes': u'True',
u'ec2_virtualization_type': u'hvm',
u'ec2_vpc_id': u'vpc-e6c17c83',
'group_names': [u'ami_2d39803a',
u'ec2',
u'i-435014bb',
u'key_albert',
u'security_group_abds_secgroup',
u'security_group_default',
u'tag_datanodes_True',
u'tag_frontendnodes_True',
u'tag_hadoopnodes_True',
u'tag_historyservernodes_True',
u'tag_journalnodes_True',
u'tag_namenodes_True',
u'tag_resourcemanagernodes_True',
u'tag_zookeepernodes_True',
u'type_t2_micro',
u'us-east-1',
u'us-east-1a',
u'vpc_id_vpc_e6c17c83'],
'groups': {'all': [u'52.23.213.103', u'54.196.41.145', u'54.209.137.235'],
u'ami_2d39803a': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'ec2': [u'52.23.213.103', u'54.196.41.145', u'54.209.137.235'],
u'i-435014bb': [u'52.23.213.103'],
u'i-e250141a': [u'54.196.41.145'],
u'i-e350141b': [u'54.209.137.235'],
u'key_albert': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'security_group_abds_secgroup': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'security_group_default': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'tag_datanodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'tag_frontendnodes_False': [u'54.196.41.145', u'54.209.137.235'],
u'tag_frontendnodes_True': [u'52.23.213.103'],
u'tag_hadoopnodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'tag_historyservernodes_False': [u'54.196.41.145',
u'54.209.137.235'],
u'tag_historyservernodes_True': [u'52.23.213.103'],
u'tag_journalnodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'tag_namenodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'tag_resourcemanagernodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'tag_zookeepernodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'type_t2_micro': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
'ungrouped': ['localhost'],
u'us-east-1': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'us-east-1a': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
u'vpc_id_vpc_e6c17c83': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235']},
'inventory_dir': '/home/albert/git/aws-cloudformation-by-ansible',
'inventory_file': 'ec2.py',
'inventory_hostname': u'52.23.213.103',
'inventory_hostname_short': u'52',
u'module_setup': True,
'omit': '__omit_place_holder__943b1cb4d91b305bcddb59adcbdfd6146fda5e7e',
'playbook_dir': u'/home/albert/git/aws-cloudformation-by-ansible'}
As you noticed that tags create separate host groups in a tag_<key>_<value>
form, for example, namenodes are listed like:
u'tag_namenodes_True': [u'52.23.213.103',
u'54.196.41.145',
u'54.209.137.235'],
It is good because Ansible directly call this group like:
$ ansible -m ping -i ec2.py -u ubuntu "tag_namenodes_True"
However, if hostnames should be just namenodes without any prefix or postfix,
we can fix that using add_host module. Let’s see that in convert.yml.
Converting Inventory Groups¶
convent.yml uses add_host to create a new inventory groups and it
corrects group names from tag_<key>_<value> fomat to <key> like just
namenodes. Sample task looks like
- name: convert EC2 tags_*_True inventory groups
hosts: "tag_namenodes_True"
tasks:
- name: Add new instance to host group (namenodes)
add_host: hostname="{{ hostvars[item]['ec2_ip_address'] }}" groupname=namenodes
with_items: "{{ groups['tag_namenodes_True'] }}"
This task runs towards tag_namenodes_True group and create a new group
named namenodes with EC2 IP addresses therefore following ansible playbooks
or roles can be used without modification in hostnames.
Terminating EC2 Instances¶
state: 'absent' terminates ec2 instances with instance ids which
boot.yml stores using ec2_facts. Therefore, terminate.yml
reads the YAML fact file and gets instance ids to terminate. include_vars
imports data from YAML and makes them accessible in Ansible Plays.
Sample ec2_fact file looks like:
"ansible_facts": {
"changed": true,
"instance_ids": [
"i-8c064074",
"i-8306407b"
],
"instances": [
{
"ami_launch_index": "0",
"architecture": "x86_64",
"block_device_mapping": {
"/dev/sda1": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-71970cf0"
}
},
"dns_name": "",
"ebs_optimized": false,
"groups": {
"sg-115b376b": "abds_secgroup",
"sg-69ebc60c": "default"
},
"hypervisor": "xen",
"id": "i-8c064074",
"image_id": "ami-2d39803a",
"instance_type": "t2.micro",
"kernel": null,
"key_name": "albert",
"launch_time": "2016-09-17T03:14:11.000Z",
"placement": "us-east-1a",
"private_dns_name": "ip-172-30-3-89.ec2.internal",
"private_ip": "172.30.3.89",
"public_dns_name": "",
"public_ip": "54.159.24.94",
"ramdisk": null,
"region": "us-east-1",
"root_device_name": "/dev/sda1",
"root_device_type": "ebs",
"state": "running",
"state_code": 16,
"tags": {
"datanodes": "True",
"frontendnodes": "False",
"hadoopnodes": "True",
"historyservernodes": "False",
"journalnodes": "True",
"namenodes": "True",
"resourcemanagernodes": "True",
"zookeepernodes": "True"
},
"tenancy": "default",
"virtualization_type": "hvm"
},
{
"ami_launch_index": "1",
"architecture": "x86_64",
"block_device_mapping": {
"/dev/sda1": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-c4960d45"
}
},
"dns_name": "",
"ebs_optimized": false,
"groups": {
"sg-115b376b": "abds_secgroup",
"sg-69ebc60c": "default"
},
"hypervisor": "xen",
"id": "i-8306407b",
"image_id": "ami-2d39803a",
"instance_type": "t2.micro",
"kernel": null,
"key_name": "albert",
"launch_time": "2016-09-17T03:14:11.000Z",
"placement": "us-east-1a",
"private_dns_name": "ip-172-30-3-88.ec2.internal",
"private_ip": "172.30.3.88",
"public_dns_name": "",
"public_ip": "54.175.124.32",
"ramdisk": null,
"region": "us-east-1",
"root_device_name": "/dev/sda1",
"root_device_type": "ebs",
"state": "running",
"state_code": 16,
"tags": {
"datanodes": "True",
"frontendnodes": "False",
"hadoopnodes": "True",
"historyservernodes": "False",
"journalnodes": "True",
"namenodes": "True",
"resourcemanagernodes": "True",
"zookeepernodes": "True"
},
"tenancy": "default",
"virtualization_type": "hvm"
}
],
"tagged_instances": [
{
"ami_launch_index": "0",
"architecture": "x86_64",
"block_device_mapping": {
"/dev/sda1": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-71970cf0"
}
},
"dns_name": "",
"ebs_optimized": false,
"groups": {
"sg-115b376b": "abds_secgroup",
"sg-69ebc60c": "default"
},
"hypervisor": "xen",
"id": "i-8c064074",
"image_id": "ami-2d39803a",
"instance_type": "t2.micro",
"kernel": null,
"key_name": "albert",
"launch_time": "2016-09-17T03:14:11.000Z",
"placement": "us-east-1a",
"private_dns_name": "ip-172-30-3-89.ec2.internal",
"private_ip": "172.30.3.89",
"public_dns_name": "",
"public_ip": "54.159.24.94",
"ramdisk": null,
"region": "us-east-1",
"root_device_name": "/dev/sda1",
"root_device_type": "ebs",
"state": "running",
"state_code": 16,
"tags": {
"datanodes": "True",
"frontendnodes": "False",
"hadoopnodes": "True",
"historyservernodes": "False",
"journalnodes": "True",
"namenodes": "True",
"resourcemanagernodes": "True",
"zookeepernodes": "True"
},
"tenancy": "default",
"virtualization_type": "hvm"
},
{
"ami_launch_index": "1",
"architecture": "x86_64",
"block_device_mapping": {
"/dev/sda1": {
"delete_on_termination": true,
"status": "attached",
"volume_id": "vol-c4960d45"
}
},
"dns_name": "",
"ebs_optimized": false,
"groups": {
"sg-115b376b": "abds_secgroup",
"sg-69ebc60c": "default"
},
"hypervisor": "xen",
"id": "i-8306407b",
"image_id": "ami-2d39803a",
"instance_type": "t2.micro",
"kernel": null,
"key_name": "albert",
"launch_time": "2016-09-17T03:14:11.000Z",
"placement": "us-east-1a",
"private_dns_name": "ip-172-30-3-88.ec2.internal",
"private_ip": "172.30.3.88",
"public_dns_name": "",
"public_ip": "54.175.124.32",
"ramdisk": null,
"region": "us-east-1",
"root_device_name": "/dev/sda1",
"root_device_type": "ebs",
"state": "running",
"state_code": 16,
"tags": {
"datanodes": "True",
"frontendnodes": "False",
"hadoopnodes": "True",
"historyservernodes": "False",
"journalnodes": "True",
"namenodes": "True",
"resourcemanagernodes": "True",
"zookeepernodes": "True"
},
"tenancy": "default",
"virtualization_type": "hvm"
}
]
},
"changed": false,
"invocation": {
"module_args": {
"file": "/tmp/a.yml"
},
"module_name": "include_vars"
}
}