title: Awk/sed replace newlines
body: <h2>Intro:</h2>
  
  <p>I have been given a CSV file in which the field delimiter is the pipe characted (i.e., <code>|</code>).
  This file has a pre-defined number of fields (say <code>N</code>). I can discover the value of <code>N</code> by reading the header of the CSV file, which we can assume to be correct.</p>
  
  <h2>Problem:</h2>
  
  <p>Some of the fields contain a newline character by mistake, which makes the line appear shorter than required (i.e., it has <code>M</code> fields, with <code>M &lt; N</code>).</p>
  
  <p>What I need to create is a <code>sh</code> script (not <code>bash</code>) to fix those lines.</p>
  
  <h2>Attempted solution:</h2>
  
  <p>I tried creating the following script to try fixing the file:</p>
  
  <pre><code>if [ $# -ne 1 ]
  then
      echo "Usage: $0 &lt;filename&gt;"
      exit
  fi
  
  # get first line
  first_line=$(head -n 1 $1)
  
  # get number of fields
  num_separators=$(echo "$first_line" | tr -d -c '|' | awk '{print length}')
  
  cat $1  | awk -v numFields=$(( num_separators + 1 )) -F '|' '
  {
      totRecords = NF/numFields
      # loop over lines
      for (record=0; record &lt; totRecords; record++) {
          output = ""
          # loop over fields
          for (i=0; i&lt;numFields; i++) {
              j = (numFields*record)+i+1 
              # replace newline with question mark
              sub("\n", "?", $j)
              output = output (i &gt; 0 ? "|" : "") $j 
          }
          print output
      }
  }
  '
  </code></pre>
  
  <p>However, the newline character is still present.
  How can I fix that problem?</p>
  
  <h2>Example of the CSV:</h2>
  
  <pre><code>FIRST_NAME|LAST_NAME|NOTES
  John|Smith|This is a field with a
  newline
  Foo|Bar|Baz
  </code></pre>
  
  <h2>Expected output:</h2>
  
  <pre><code>FIRST_NAME|LAST_NAME|NOTES
  John|Smith|This is a field with a * newline
  Foo|Bar|Baz
  
  * I don't care about the replacement, it could be a space, a question mark, whatever except a newline or a pipe (which would create a new field)
  </code></pre>