title: Apache Spark Native Libraries
body: <p>I was recently able to build Apache Hadoop 2.5.1 with native 64 bit support. So,  I got rid of the annoying Native Libraries Warning. </p>
  
  <p>I'm trying to configure Apache Spark. When I start spark-shell, the same warning appears: </p>
  
  <pre><code>14/09/14 18:48:42 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
  </code></pre>
  
  <p>Some tips:</p>
  
  <p>I had to download a pre-built 2.4 version of Spark because there is still no profile for Hadoop 2.5 with Maven.</p>
  
  <p>The following exports were added to spark-env.sh:</p>
  
  <pre><code>export HADOOP_CONF_DIR=/opt/hadoop-2.5.1/etc/hadoop
  
  export SPARK_LIBRARY_PATH=/opt/hadoop-2.5.1/lib/native
  </code></pre>
  
  <p>Didn't work with spark-shell and spark-submit. My Hadoop local installation is configured as pseudo-distributed (ResourceManager + YARN support).</p>