title: Reading a file in Spark in cluster mode in Amazon EC2
body: <p>I'm trying to execute a spark program in cluster mode in Amazon Ec2 using</p>
  
  <pre><code>spark-submit --master spark://&lt;master-ip&gt;:7077 --deploy-mode cluster --class com.mycompany.SimpleApp ./spark.jar
  </code></pre>
  
  <p>And the class has a line that tries to read a file:</p>
  
  <pre><code>JavaRDD&lt;String&gt; logData = sc.textFile("/user/input/CHANGES.txt").cache();
  </code></pre>
  
  <p>I'm unable to read this txt file in cluster mode even if I'm able to read in standalone mode. In cluster mode, it's looking to read from hdfs. So I put the file in hdfs at /root/persistent-hdfs using </p>
  
  <pre><code>hadoop fs -mkdir -p /wordcount/input
  hadoop fs -put /app/hadoop/tmp/input.txt /wordcount/input/input.txt
  </code></pre>
  
  <p>And I can see the file using <code>hadoop fs -ls /workcount/input</code>. But Spark is still unable to read the file. Any idea what I'm doing wrong. Thanks.</p>