title: Issue with BinaryReader.ReadChars()
body: <p>I've run into what I believe is an issue with the BinaryReader.ReadChars() method.  When I wrap a BinaryReader around a raw socket NetworkStream occasionally I get a stream corruption where the stream being read gets out of sync.  The stream in question contains messages in a binary serialisation protocol.</p>
  
  <p>I've tracked this down to the following</p>
  
  <ul>
  <li>It only happens when reading a unicode string (encoded using the Encoding.BigEndian)</li>
  <li>It only happens when the string in question is split across two tcp packets (confirmed using wireshark)</li>
  </ul>
  
  <p>I think what is happening is the following (in the context of the example below)</p>
  
  <ul>
  <li>BinaryReader.ReadChars() is called asking it to read 3 characters (string lengths are encoded before the string itself)</li>
  <li>First loop internally requests a read of 6 bytes (3 remaining characters * 2 bytes/char) off the network stream</li>
  <li>Network stream only has 3 bytes available</li>
  <li>3 bytes read into local buffer</li>
  <li>Buffer handed to Decoder</li>
  <li>Decoder decodes 1 char, and keeps the other byte in it's own internal buffer</li>
  <li>Second loop internally requests a read of 4 bytes! (2 remaining characters * 2 bytes/char)</li>
  <li>Network stream has all 4 bytes available</li>
  <li>4 bytes read into local buffer</li>
  <li>Buffer handed to Decoder</li>
  <li>Decoder decodes 2 char, and keeps the remaining 4th bytes internally</li>
  <li>String decode is complete</li>
  <li><p>Serialisation code attempts to unmarshal the next item and croaks because of stream corruption.</p>
  
  <pre><code>char[] buffer = new char[3];
  int charIndex = 0;
  
  Decoder decoder = Encoding.BigEndianUnicode.GetDecoder();
  
  // pretend 3 of the 6 bytes arrives in one packet
  byte[] b1 = new byte[] { 0, 83, 0 };
  int charsRead = decoder.GetChars(b1, 0, 3, buffer, charIndex);
  charIndex += charsRead;
  
  // pretend the remaining 3 bytes plus a final byte, for something unrelated,
  // arrive next
  byte[] b2 = new byte[] { 71, 0, 114, 3 };
  charsRead = decoder.GetChars(b2, 0, 4, buffer, charIndex);
  charIndex += charsRead;
  </code></pre></li>
  </ul>
  
  <p>I think the root is a bug in the .NET code which uses charsRemaining * bytes/char each loop to calculate the remaining bytes required.  Because of the extra byte hidden in the Decoder this calculation can be off by one causing an extra byte to be consumed off the input stream.</p>
  
  <p>Here's the .NET framework code in question</p>
  
  <pre><code>    while (charsRemaining&gt;0) { 
          // We really want to know what the minimum number of bytes per char 
          // is for our encoding.  Otherwise for UnicodeEncoding we'd have to
          // do ~1+log(n) reads to read n characters. 
          numBytes = charsRemaining;
          if (m_2BytesPerChar)
              numBytes &lt;&lt;= 1;
  
          numBytes = m_stream.Read(m_charBytes, 0, numBytes);
          if (numBytes==0) { 
              return (count - charsRemaining); 
          } 
          charsRead = m_decoder.GetChars(m_charBytes, 0, numBytes, buffer, index);
  
          charsRemaining -= charsRead;
          index+=charsRead;
      }
  </code></pre>
  
  <p>I'm not entirely sure if this is a bug or just a misuse of the API.  To work round this issue I'm just calculating the bytes required myself, reading them, and then running the byte[] through the relevant Encoding.GetString().  However this wouldn't work for something like UTF-8.</p>
  
  <p>Be interested to hear people's thoughts on this and whether I'm doing something wrong or not.   And maybe it will save the next person a few hours/days of tedious debugging.</p>
  
  <p>EDIT: posted to connect <a href="http://connect.microsoft.com/VisualStudio/feedback/ViewFeedback.aspx?FeedbackID=514921" rel="nofollow noreferrer">Connect tracking item</a></p>