How To Configure Big Data Hadoop In Just A Few Steps Using Ansible Automated Script

Hey guys hope you all are doing good today in this article we are going to set the Hadoop name node and data node with the power of ansible.

If you have not checked my previous article on ansible I recommend you to read that first in that we have talked about how to configure yum that is necessary for any practical and in that we have also covered how to automate docker using ansible.

First, we have to set our inventory file for that we have made two groups controller and data so that when we run playbook we can specify the groups because we don’t want our data node server to set up as a name node. Once the inventory is setup repeat the same steps to set up an ansible configuration file and checking connectivity as shown in the above article.

Inventory

Once All the steps are done make a new folder for setting name node from the command:-

mkdir/folername

Core-site.xml

For setting up the name node we have to edit two files hdfs-site.xml and core-site.xml so we have to make a clone of those files in our ansible controller node.

we have use the jinja template inside the core-site.xml file.

we also need Hadoop and JDK installation file so that we can copy in target node and install them.

hdfs-site.xml
Deploying playbook

Once we are done with all required files we can deploy our playbook.

As you can see below we have successfully set up the name node

Name node running

All the playbooks will be in GitHub attached at the end and the installation file in the drive link attached at the end.😎

Once done with setting up the name node now let's move to set up the data node first create a new folder to setup datanode

In that folder, we need the same files as we required to set up in name node only change which we have to to do that is in hdfs-site.xml file that is shown below

hdfs-sire.xml file
Datanode started
Datanode connected

Guys, here we come to the end of this blog I hope you all like it and found it informative. If have any query feel free to reach me :)

Github link:-https://github.com/guptaadi123/Ansible.git

Drive link:

https://drive.google.com/drive/folders/152YHK4LnRiMk4Kup00GhclbtQje3z39l?usp=sharing

Guys follow me for such amazing blogs and if have any review then please let me know I will keep those points in my mind next time while writing blogs. If want to read more such blog to know more about me here is my website link https://sites.google.com/view/adityvgupta/home.Guys Please do not hesitate to keep 👏👏👏👏👏 for it (An Open Secret: You can clap up to 50 times for a post, and the best part is, it wouldn’t cost you anything), also feel free to share it across. This really means a lot to me.

--

--

--

Love podcasts or audiobooks? Learn on the go with our new app.

Recommended from Medium

Java coding for kids.

Making the simple things simple: Coda filters

Inventory Slot Numbers Minecraft

Numbers

How to Automate and Scale SQL capabilities using Google Cloud Platform

Domain-driven Design in Android — Part 2

Nakama: A year in review

A Dialogue about Value

Lockdown SmackDown! — Get Zel

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store
Gupta Aditya

Gupta Aditya

More from Medium

Secure AWS to Azure integration

AWS cloud with Robotic Process Automation

Airflow — Build, Host, Maintain Your Own Dependencies with GitHub and Docker Hub (for free)

Google Cloud Database