Steps to Configure Hadoop and start cluster services using Ansible Playbook
After That we will Create our inventory file which contains IP address of Datanode and Namenode,

Now, we will create Ansible configuration file ansible.cfg ,

After that we will create Ansible Playbook to configure NameNode or MasterNode

- hosts: NameNode

vars:

jdk: jdk-8u171-linux-x64.rpm

hadoop: hadoop-1.2.1–1.x86_64.rpm tasks:

— name: “Copy jdk file”

copy:

dest: “/root/”

src: “/home/ec2-user/{{ jdk }}” - name: “Copy hadoop file”

copy:

dest: “/root/”

src: “/home/ec2-user/{{ hadoop }}” - name: “Install jdk”

command: “rpm -ivh {{ jdk }}” - name: “Install hadoop”

command: “rpm -ivh {{ hadoop }} — force” - name: “Directory Creation”

file:

path: “/root/namenode”

state: directory - name: “Copy core file”

copy:

dest: “/etc/hadoop”

src: “/ansibleWS/Namenode/core-site.xml” - name: “Copy hdfs file”

copy:

dest: “/etc/hadoop”

src: “/ansibleWS/Namenode/hdfs-site.xml” - name: “Formating NameNode”

shell: echo Y | hadoop namenode -format - name: “Start Hadoop Service”

command: “hadoop-daemon.sh start namenode"

After, our NameNode playbook is made we will run using following command:

$ansible-playbook hadoopNN.yml

Valla! Our namenode is configured now its time to make a playbook for DataNode, Let’s do that,

- hosts: DataNode

vars:

jdk: jdk-8u171-linux-x64.rpm

hadoop: hadoop-1.2.1–1.x86_64.rpm tasks:

— name: “Copy jdk file”

copy:

dest: “/root/”

src: “/home/ec2-user/{{ jdk }}” - name: “Copy hadoop file”

copy:

dest: “/root/”

src: “/home/ec2-user/{{ hadoop }}” - name: “Install jdk”

command: “rpm -ivh {{ jdk }}”

ignore_errors: yes - name: “Install hadoop”

command: “rpm -ivh {{ hadoop }} — force”

ignore_errors: yes - name: “Directory Creation”

file:

path: “/root/datanode”

state: directory - name: “Copy core file”

copy:

dest: “/etc/hadoop”

src: “/ansibleWS/Datanode/core-site.xml” - name: “Copy hdfs file”

copy:

dest: “/etc/hadoop”

src: “/ansibleWS/Datanode/hdfs-site.xml” - name: “Start Datanode Service”

command: “hadoop-daemon.sh start datanode”

Running the ansible DataNode playbook:

$ansible-playbook hadoopDN.yml

Hurrahh!! Now our Data node is also configured, let’s see our DataNode is connected to NameNode or not:

$hadoop dfsadmin -report

Finally, We can see that we are able to our Hadoop Cluster is Ready with Automation.
[]
['Ansible', 'Hadoop', 'Arth', 'Vimal Daga', 'Righteducation']