Apache Hadoop is a strong framework based on open sources that is capable of implementation of distributed storage and processing of massive data volumes across a system made up of a network of computers. It is the favoured technology for the big data steel Industry, that is, its scalability, reliability, and fault-protecting characteristics are superior. Let's take you through the Hadoop installation process on a Mac OS computer using this guide.
Table of Content
Installation Prerequisites
Before we proceed, ensure that the necessary conditions below are met on your macOS system:
1. Java Development Kit (JDK): Hadoop is written in Java and therefore needs it to function. Ensure that you have the JDK 8 or higher packed and ready for installation on your computer. You can get JDK either using a website Oracle (as an option) or a HomeBrew project, as brew installs OpenJDK.
2. SSH: Hadoop SSH communication is tied to Hadoop's ability to transmit data between different nodes in the cluster. macOS usually comes with SSH installed by default, however, to enable the feature you will need to navigate to System Preferences -> Sharing -> Remote Login.
3. Homebrew: Homebrew, the package manager for macOS, lists software components in their preferred software location. With Homebrew not being installed, use the following command on your shell.
4. the Environment Setup: Make sure that the environment variable is correctly set up. This responsibility can be taken up by editing the ~/.bash_profile or ~/.zshrc file and adding in the mentioned lines.
- Run the replace /path/to/your/java/home and replace /path/to/your/Hadoopwith/home with your full path to the installed Java and Hadoop.
How to Install Hadoop on Mac?
Apache Hadoop is a strong framework based on open sources and we need to follow the below-mentioned easy steps to download it properly on MacOS.
Step 1: Downloading Hadoop
You can download the latest stable version of Hadoop from the Apache Hadoop website. Select the one that fits your requirements and download the tarball (tar.gz) from the server to your machine.
Then, download the files, then extract the file into any of the directories that you wish.
For example
Step 2: Configuration
Navigate to the Hadoop configuration directory (in $HADOOP_HOME/etc/Hadoop) and modify the given configuration as indicated below.
a. core-site.xml:
b. hdfs-site.xml:
c. mapred-site.xml (if it doesn't exist, create it):
d. yarn-site.xml:
Step 3: Formatting HDFS
The prerequisite for starting Hadoop services is formatting the disk HDFS (Hadoop Distributed File System) by the end of it.
Run the following command
Step 4: Start Hadoop Services.
Launch Hadoop services by executing the commands mentioned below.
a. Start HDFS:
b. Start YARN:
Step 5: Verify Installation
Hadoop is a program that you can check its correct running by viewing the Hadoop web interface. Open your web browser and go to http://localhost:9876 for the HDFS NameNode interface and http://localhost:6060 for the YARN ResourceManager Interface.
Conclusion
Congratulations! You have successfully installed Apache Hadoop on macOS. You can now move on to the full spectrum of possibilities afforded by Hadoop and start investigating its role in distributed data processing and analysis. Keep in mind that the official Hadoop documentation can be of help to those interested in more advanced configurations and more complex deployments.
