title: Using a text file as Spark streaming source for testing purpose
body: <p>I want to write a test for my spark streaming application that consume a flume source.</p>
  
  <p><a href="http://mkuthan.github.io/blog/2015/03/01/spark-unit-testing/" rel="noreferrer">http://mkuthan.github.io/blog/2015/03/01/spark-unit-testing/</a> suggests using ManualClock but for the moment reading a file and verifying outputs would be enough for me.</p>
  
  <p>So I wish to use :</p>
  
  <pre><code>JavaStreamingContext streamingContext = ...
  JavaDStream&lt;String&gt; stream = streamingContext.textFileStream(dataDirectory);
  stream.print();
  streamingContext.awaitTermination();
  streamingContext.start();
  </code></pre>
  
  <p>Unfortunately it does not print anything.</p>
  
  <p>I tried:</p>
  
  <ul>
  <li>dataDirectory = "hdfs://node:port/absolute/path/on/hdfs/"</li>
  <li>dataDirectory = "file://C:\\absolute\\path\\on\\windows\\"</li>
  <li>adding the text file in the directory BEFORE the program begins</li>
  <li>adding the text file in the directory WHILE the program run</li>
  </ul>
  
  <p>Nothing works.</p>
  
  <p>Any suggestion to read from text file?</p>
  
  <p>Thanks,</p>
  
  <p>Martin </p>