title: how to quieten output from spark-submit?
body: <p>I'm trying to automate spark-submit jobs for spark on bluemix, but I'm receiving a lot of output (e.g. file upload status).  This verbose output is fine for manually running spark-submit, but it is just noise when trying to execute if from another script.</p>
  
  <p>Is there a way to quieten the output from the <strong>bluemix custom spark-submit script</strong>?</p>
  
  <hr>
  
  <p><strong>Update:</strong></p>
  
  <p>The output I'm trying to quieten is that generated from the spark-submit script, for example:</p>
  
  <pre><code>Uploading ./truststore.jar
    % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                   Dload  Upload   Total   Spent    Left  Speed
  100  3103    0    99  100  3004     60   1835  0:00:01  0:00:01 --:--:--  1835
  </code></pre>
  
  <p>I am <em>not</em> trying to quieten the output from spark, so quietening <code>log4j</code> will not help.</p>