title: How to prevent "Execution failed:[Errno 32] Broken pipe" in Airflow
body: <p>I just started using Airflow to coordinate our ETL pipeline.</p>
  
  <p>I encountered the pipe error when I run a dag.</p>
  
  <p>I've seen a general stackoverflow discussion <a href="https://stackoverflow.com/questions/11866792/how-to-prevent-errno-32-broken-pipe">here</a>.</p>
  
  <p>My case is more on the Airflow side. According to the discussion in that post, the possible root cause is:</p>
  
  <blockquote>
    <p>The broken pipe error usually occurs if your request is blocked or
    takes too long and after request-side timeout, it'll close the
    connection and then, when the respond-side (server) tries to write to
    the socket, it will throw a pipe broken error.</p>
  </blockquote>
  
  <p>This might be the real cause in my case, I have a pythonoperator that will start another job outside of Airflow, and that job could be very lengthy (i.e. 10+ hours), I wonder if what is the mechanism in place in Airflow that I can leverage to prevent this error.</p>
  
  <p>Can anyone help? </p>
  
  <p>UPDATE1 20190303-1:</p>
  
  <p>Thanks to @y2k-shubham for the SSHOperator, I am able to use it to set up a SSH connection successfully and am able to run some simple commands on the remote site (indeed the default ssh connection has to be set to localhost because the job is on the localhost) and am able to see the correct result of <code>hostname</code>, <code>pwd</code>.</p>
  
  <p>However, when I attempted to run the actual job, I received same error, again, the error is from the jpipeline ob instead of the Airflow dag/task.</p>
  
  <p>UPDATE2: 20190303-2</p>
  
  <p>I had a successful run (airflow test) with no error, and then followed another failed run (scheduler) with same error from pipeline. </p>