title: using EOF for signaling on unnamed pipes
body: <p>I have a test program that uses unnamed pipes created with pipe() to communicate between parent and child processes created with fork() on a Linux system.</p>
  
  <p>Normally, when the sending process closes the write fd of the pipe, the receiving process returns from read() with a value of 0, indicating EOF.</p>
  
  <p>However, it seems that if I stuff the pipe with a fairly large amount of data (maybe 100K bytes0 before the receiver starts reading, the receiver blocks after reading all the data in the pipe - even though the sender has closed it.</p>
  
  <p>I have verified that the sending process has closed the pipe with lsof, and it seems pretty clear that the receiver is blocked.</p>
  
  <p>Which leads to the question: is closing one end of the pipe a reliable way to let the receiver know that there is no more data?</p>
  
  <p>If it is, and there are no conditions that can lead to a read() blocking on an empty, closed FIFO, there's something wrong with my code. If not, it means I need to find an alternate method of signalling the end of the data stream.</p>
  
  <p><strong>Resolution</strong></p>
  
  <p>I was pretty sure that the original assumption was correct, that closing a pipe causes an EOF at the reader side, this question was just a shot in the dark - thinking maybe there was some subtle pipe behavior I was overlooking. Nearly every example you ever see with pipes is a toy that sends a few bytes and exits. Things often work differently when you are no longer doing atomic operations.</p>
  
  <p>In any case, I tried to simplify my code to flush out the problem and was successful in finding my problem. In pseudocode, I ended up doing something like this:</p>
  
  <pre><code>create pipe1
  if ( !fork() ) {
      close pipe1 write fd
     do some stuff reading pipe1 until EOF
  }
  create pipe2
  if ( !fork() )  {
     close pipe2 write fd
     do some stuff reading pipe2 until EOF
  }
  close pipe1 read fd
  close pipe2 read fd
  write data to pipe1
  get completion response from child 1
  close pipe1 write fd
  write data to pipe2
  get completion response from child 2
  close pipe2 write fd
  wait for children to exit
  </code></pre>
  
  <p>The child process reading pipe1 was hanging, but only when the amount of data in the pipe became substantial. This was occurring even though I had closed the pipe that child1 was reading.</p>
  
  <p>A look at the source shows the problem. When I forked the second child process, it grabbed its own copy of the pipe1 file descriptors, which were left open. Even though only one process should be writing to the pipe, having it open in the second process kept it from going into an EOF state.</p>
  
  <p>The problem didn't show up with small data sets, because child2 was finishing its business quickly, and exiting.  But with larger data sets, child2 wasn't returning quickly, and I ended up with a deadlock.</p>