title: Oozie spark action Log4j configuration
body: <p>I am working on <code>Oozie</code>, using a <code>Spark</code> action on a <code>Hortonworks2.5</code> cluster. I have configured this job in <code>yarn client mode</code>, with <code>master=yarn mode=client</code>. 
  My <code>log4j</code> configuration is shown below. </p>
  
  <pre><code>log4j.appender.RollingAppender.File=/opt/appName/logs/appNameInfo.log
  log4j.appender.debugFileAppender.File=/opt/appName/logs/appNameDebug.log
  log4j.appender.errorFileAppender.File=/opt/appName/logs/appNameError.log
  </code></pre>
  
  <p>The job expectation is once we trigger <code>oozie</code> job, in the above locations we should be able to see my application logs as <code>Info,Debug,Error</code> respectively.</p>
  
  <p>Below is my <code>spark-opts</code> tag in my <code>workflow.xml</code></p>
  
  <pre><code>&lt;spark-opts&gt;--driver-memory 4G --executor-memory 4G --num-executors 6 --executor-cores 3 --files /tmp/logs/appName/log4j.properties --conf spark.driver.extraJavaOptions='-Dlog4j.configuration=file:/tmp/logs/appName/log4j.properties' --conf spark.executor.extraJavaOptions='-Dlog4j.configuration=file:/tmp/logs/appName/log4j.properties'&lt;/spark-opts&gt; 
  </code></pre>
  
  <p>Once I trigger <code>oozie</code> coordinator, I am not able to see my application logs in <code>/opt/appName/logs/</code> as configured in <code>log4j.properties</code>.</p>
  
  <p>The same configuration is working with plain <code>Spark-submit</code> when I run from the node where <code>/tmp/logs/appName/log4j.properties</code> available in that particular node. Can some one please look in to the issue. It is not able to write to the location which is configured in <code>log4j.properties</code> file.</p>
  
  <p>Is this <code>log4j.properties</code> file should be in <code>hdfs</code>?? if so, how to provide in <code>spark-opts</code>. is it would be <code>hdfs://</code> ??</p>
  
  <p>Can some one look in to the issue please?</p>