title: hadoop Task Side-Effect Files
body: <p>I'm using hadoop with 0 Reduces. The goal is to create an object incrementally in the <code>map</code> method. Then at some point write (serialize) it to the output folder. Like I said the <code>reduce</code> piece won't do anything here. How do I do this? This is what I have:</p>
  
  <p>In the configure method I get the path to the file:</p>
  
  <pre><code>@Override
  public void configure(JobConf conf) {      
    taskSideEffectFile = FileOutputFormat.getWorkOutputPath(conf) + "/temp";    
  }
  </code></pre>
  
  <p>In the map method I'm building my object and eventually I would like to serialize it, for now I'm trying to write it always on the map method:</p>
  
  <pre><code>@Override
  public void map(LongWritable key, Text value,
      OutputCollector&lt;Text, IntWritable&gt; output, Reporter reporter)
      throws IOException {        
  
    AddInstanceToClassifier(value.toString());
  
    try
    {             
      //serialize classifier
      weka.core.SerializationHelper.write( taskSideEffectFile, nb);
  
    }
    catch (Exception ex)
    {
      System.err.println("Failed to serialize classifier: " + ex.getMessage());
      throw new IOException("taskSideEffectFile: " + ex.getMessage());
  
    } 
  
  }
  </code></pre>
  
  <p>This is the error I'm getting:</p>
  
  <pre><code>12/05/09 22:47:00 INFO mapred.JobClient:  map 0% reduce 0%
  12/05/09 22:47:08 INFO mapred.JobClient: Task Id : attempt_201205091117_0015_m_000001_0, Status : FAILED
  java.io.IOException: taskSideEffectFile: hdfs:/192.168.78.129:9000/user/hadoop-user/output/_temporary/_attempt_201205091117_0015_m_000001_0/temp (No such file or directory)
      at naive.bayes.hadoop.MusicClassifierMapper.SaveClassifier(MusicClassifierMapper.java:168)
      at naive.bayes.hadoop.MusicClassifierMapper.map(MusicClassifierMapper.java:121)
      at naive.bayes.hadoop.MusicClassifierMapper.map(MusicClassifierMapper.java:1)
      at org.apache.hadoop.mapred.MapRunner.run(MapRunner.java:47)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:227)
      at org.apache.hadoop.mapred.TaskTracker$Child.main(TaskTracker.java:2209)
  </code></pre>
  
  <p><strong>Note</strong>:I'm using yahoo's hadoop-0.18.0 (I saw this as my only way to run the apps from eclipse)</p>