title: Multi-Byte Character Support over TServerSocket Delphi
body: <p>While working on a multi-user chat application I've got stuck around getting the multi-byte chars to work over TServerSocket / TClientSocket.</p>
  
  <p>This is the part where the client sends the message to the server:</p>
  
  <pre><code>  procedure TChatForm.SendBtnClick(Sender: TObject);
    var str : string;
    begin
      str := MsgLabel.Text;
      ClientSocket.Socket.SendText('message' + separator + nickname + separator + str);
      MsgLabel.Text := '';
      add_text(MsgBox,MsgLabel,nickname+': '+str,'none');
    end;
  </code></pre>
  
  <p>This is how the server parses the received data:</p>
  
  <pre><code>procedure TServerForm.ServerSocketClientRead(Sender: TObject;Socket: TCustomWinSocket);
  var
    i,hnd : Integer;
    recv : string;
    arr : TStringArray; // type TStringArray = array of string;
  begin
    recv := Socket.ReceiveText;
    hnd := Socket.Handle; //using this to avoid sending received data back to the client
    arr := SplitStr(recv,separator); 
    //SplitStr is a function i use because TStringList.DelimitedText uses only a char as delimiter 
  
    // sending the data to the others users / but the expeditor - async error workaround
    for i:=0 to ServerSocket.Socket.ActiveConnections-1 do begin
      if ServerSocket.Socket.Connections[i].Handle &lt;&gt; hnd then
        ServerSocket.Socket.Connections[i].SendText(recv);
    end;
  
    if arr[0] = 'connect' then begin
  
      // adding the connected user to the tlistbox
      Contacts.Items.Add(arr[1]);
      // adding the connected message in the trichedit
      add_text(MsgBox,SendMsg,arr[1]+' has connected !','green');
  
    end else if arr[0] = 'disconnect' then begin
  
      // removing the user from the online user list
      Contacts.Items.Delete(Contacts.Items.IndexOf(arr[1]));
      // adding the disconnected message in trichedit
      add_text(MsgBox,SendMsg,arr[1]+' has disconnected !','red');
  
    end else if arr[0] = 'message' then begin
  
      // finally adding the message that user send in the TRichEdit
      add_text(MsgBox,SendMsg,arr[1]+': '+arr[2],'none');
  
    end;
  end;
  </code></pre>
  
  <p>An example of how the Socket.ReceiveText looks like: </p>
  
  <pre><code>- when user connects he sends the next message - connect^SEPARATOR^username
  - when a user sends a message - message^SEPARATOR^username^SEPARATOR^message_body
  </code></pre>
  
  <p>The structure is ACTION + SEPARATOR + USERNAME + EXTRA_DATA, thas my way of "keeping" the online users list updated. I'm new to delphi, if there's any easier way of doing that, let me know.</p>
  
  <p>The problem is now, if I'm sending multibyte characters over to the users and back, those multibyte chars are received as question marks "?".
  - "ț or ș" becomes "? or ?"</p>
  
  <p>Printscreen here: <img src="https://i.stack.imgur.com/xXdz9.png" alt="enter image description here"></p>
  
  <p>EDIT2: Ok, after all the changes have been made, thanks to your answers, I bumped into a problem while trying to send the data received by the server from the client back to the other clients. Well this problem has 2 little bumps:</p>
  
  <ol>
  <li><p>This is how the server sends a "global" message to the users.</p>
  
  <pre><code>procedure TServerForm.SendBtnClick(Sender: TObject);
  var
    i : Integer;
    str : String;
  begin
    str := SendMsg.Text;
  
    with ServerSocket.Socket do
    begin
      for i := 0 to ActiveConnections-1 do
        SendString(Connections[i], TSocketBuffers(Connections[i].Data).OutBuffer, 'global' + separator + str);
    end;
  
    add_text(MsgBox,SendMsg,str,'none');
    SendMsg.Text := '';
  end;
  </code></pre></li>
  <li><p>This is how server sends back to other active connections the data received from one client:</p>
  
  <pre><code>procedure TServerForm.ServerSocketClientRead(Sender: TObject;Socket: TCustomWinSocket);
  var
    Buffers: TSocketBuffers;
    i: Integer;
    RecvStr : String;
    arr : TStringArray;
  begin
    Buffers := TSocketBuffers(Socket.Data);
    if not Buffers.ReadInData(Socket) then Exit;
    Buffers.InBuffer.Position := 0;
    try
      while ReadString(Buffers.InBuffer, RecvStr) do
      begin
        arr := SplitStr(RecvStr, separator);
  
        with ServerSocket.Socket do
        begin
          for i := 0 to ActiveConnections-1 do
          begin
            if Connections[i] &lt;&gt; Socket then
              SendString(Connections[i], TSocketBuffers(Connections[i].Data).OutBuffer, arr[0]);
          end;
        end;
  
        // [ .. some string processing stuff .. ]
      end;
    finally
      CompactBuffer(Buffers.InBuffer);
    end;
  end;
  </code></pre></li>
  </ol>
  
  <p>Now, if these 2 methods are correct, then the problem is the reading data on the client side, and this is how the data is parsed on the client side following the same principle as ServerSocketClientRead(Sender: TObject;Socket: TCustomWinSocket);</p>
  
  <pre><code>    procedure TChatForm.ClientSocketRead(Sender: TObject; Socket: TCustomWinSocket);
      var
        Buffers: TSocketBuffers;
        i: Integer;
        RecvStr : String;
        arr : TStringArray;
      begin
        Buffers := TSocketBuffers(Socket.Data);
        if not Buffers.ReadInData(Socket) then Exit;
        Buffers.InBuffer.Position := 0;
        try
          while ReadString(Buffers.InBuffer, RecvStr) do begin
            ShowMessage(RecvStr); // testing if anything is received
            // [[.. some string processing code ..]]
          end;
        finally
          CompactBuffer(Buffers.InBuffer);
        end;
      end;
  </code></pre>
  
  <p><img src="https://i.stack.imgur.com/W768I.png" alt="enter image description here"></p>
  
  <p>Trying to send data from client to server works flawlessly as you can see in the image (above) string is interpreted as it should be. The problem is either trying to send the data back to the clients in ServerSocketClientRead method, either in the ClientSocketRead method. </p>
  
  <p>UPDATE 3: So I had launched the client on another pc and the problem seems to be at the ClientSocketRead method (if the ServerSocketClientRead -> SendString and the global SendBtn -> SendString are correct); I'll keep updating if any new details are found.</p>