title: Yarn timeline server log aggregation
body: <p>Configuring hadoop 2.7.1 to retain yarn jobs for longer</p>
  
  <p>Have enabled log aggregation and the jobhistory/timeline server and when a job is complete in the resource manager it does show up in the jobhistory server(if you give the correct url), however the jobhistory server is only listing M/R jobs, not yarn applications </p>
  
  <p>The problem is the job is not visible in the timeline server, in fact no jobs show in the timeline server</p>
  
  <p>Current yarn-site.xml configuration : </p>
  
  <pre><code>&lt;property&gt;
      &lt;name&gt;yarn.timeline-service.hostname&lt;/name&gt;
      &lt;value&gt;host1&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
      &lt;name&gt;yarn.timeline-service.address&lt;/name&gt;
      &lt;value&gt;${yarn.timeline-service.hostname}:10200&lt;/value&gt;
  &lt;/property&gt;
  &lt;property&gt;
      &lt;name&gt;yarn.timeline-service.webapp.address&lt;/name&gt;
      &lt;value&gt;${yarn.timeline-service.hostname}:8188&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.timeline-service.enabled&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.timeline-service.generic-application-history.enabled&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.log.server.url&lt;/name&gt;
      &lt;value&gt;http://${yarn.timeline-service.hostname}:19888/jobhistory/logs/&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.log-aggregation-enable&lt;/name&gt;
      &lt;value&gt;true&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.log-dirs&lt;/name&gt;
      &lt;value&gt;/var/vm/apps/hadoop/logs&lt;/value&gt;
  &lt;/property&gt;
  
  &lt;property&gt;
      &lt;name&gt;yarn.nodemanager.remote-app-log-dir&lt;/name&gt;
      &lt;value&gt;/var/vm/apps/hadoop/logs&lt;/value&gt;
  &lt;/property&gt;
  </code></pre>
  
  <p>Am I providing conflicting configuration in using the jobhistory server AND the timeline server?</p>
  
  <p>At the end of the day I want the yarn logs persisted to hdfs for viewing in the web-ui over the following days/weeks</p>