How to find HADOOP_HOME path on Linux?

I am trying to run the below java code on a hadoop server.

javac -classpath ${HADOOP_HOME}/hadoop-${HADOOP_VERSION}-core.jar -d wordcount_classes WordCount.java

but I am not able to locate {HADOOP_HOME}. I tried with hadoop -classpath but it is giving output as below:

/etc/hadoop/conf:/usr/lib/hadoop/lib/*:/usr/lib/hadoop/.//*:/usr/lib/hadoop-hdfs/./:/usr/lib/hadoop-hdfs/lib/*:/usr/lib/hadoop-hdfs/.//*:/usr/lib/hadoop-yarn/.//*:/usr/lib/hadoop-0.20-mapreduce/./:/usr/lib/hadoop-0.20-mapreduce/lib/*:/usr/lib/hadoop-0.20-mapreduce/.//*

Anyone has any idea about this?

2

Best Answer

Navigate to the path where hadoop is installed. locate ${HADOOP_HOME}/etc/hadoop, e.g.

/usr/lib/hadoop-2.2.0/etc/hadoop

When you type the ls for this folder you should see all these files.

capacity-scheduler.xml httpfs-site.xmlconfiguration.xsl log4j.propertiescontainer-executor.cfg mapred-env.cmdcore-site.xml mapred-env.shcore-site.xml~ mapred-queues.xml.templatehadoop-env.cmd mapred-site.xmlhadoop-env.sh mapred-site.xml~hadoop-env.sh~ mapred-site.xml.templatehadoop-metrics2.properties slaveshadoop-metrics.properties ssl-client.xml.examplehadoop-policy.xml ssl-server.xml.examplehdfs-site.xml yarn-env.cmdhdfs-site.xml~ yarn-env.shhttpfs-env.sh yarn-site.xmlhttpfs-log4j.properties yarn-site.xml~httpfs-signature.secret

Core configuration settings are available in hadoop-env.sh.

You can see classpath settings in this file and I copied some sample here for your reference.

# The java implementation to use.export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_67# The jsvc implementation to use. Jsvc is required to run secure datanodes.#export JSVC_HOME=${JSVC_HOME}export HADOOP_CONF_DIR=${HADOOP_CONF_DIR}# Extra Java CLASSPATH elements. Automatically insert capacity-scheduler.for f in $HADOOP_HOME/contrib/capacity-scheduler/*.jar; doexport HADOOP_CLASSPATH=${HADOOP_CLASSPATH+$HADOOP_CLASSPATH:}$fdone

Hope this helps!

hadoop-core jar file is in ${HADOOP_HOME}/share/hadoop/common directory, not in ${HADOOP_HOME} directory.

You can set the environment variable in your .bashrc file.

vim ~/.bashrc

Then add the following line to the end of .bashrc file.

export HADOOP_HOME=/your/hadoop/installation/directory

Just replace the path with your hadoop installation path.

Random Posts

How can I use numpy.correlate to do autocorrelation?How can I differentiate a 0 from false in a map? [duplicate]How to properly use scandir() in c?Descriptive stats from stat.desc() is in scientific notation formHow to install Python packages for SpyderAppending an id to a list if not already present in the listjQuery Scroll to DivGet item from bs4.element.TagHow to do one-liner if else statement? [duplicate]Apple Worldwide Developer Relations Certification Authority Intermediate Certificate and iOSIs it possible to download my app preview images from iTunesConnect?Decoding base64 in batchCan't use std::format in c++20How to find which points intersect with a polygon in geopandas?ESLint Parsing error: Unexpected tokenVisual Studio 2019 completely freezes when going down to data flow levelHow to reference source.yml in a model within a folder in DBTTurning off Ambient Mode in MI phonesMySQL connection pools in Node.js (mysql2)Binary Tree Java Error