Amazon Ion – A richly-typed, self-describing, hierarchical serialization format
This reminded me of a tight-packed binary format we used in the trading systems domain almost 20 years ago. Instead of including metadata/field names in each message, it had a central message dictionary that every client/server would first download a copy from. Messages had only type IDs, followed by binary packed data in the correct field order. Because of microsecond latency requirements, we even avoided the serialization/deserialization process by making the memory format of the message and the wire format one and the same. The message class contained the same buffer that you would send/store. The GetInt(fieldID) method of the class simply points to the right place in the buffer and does a cast to int. Application logs contained these messages, rather than plain text. There was a special reader to read logs. Messages were exchanged over raw TCP. They contained their own application layer sequence number so that streams could resume after disconnection.<p>In that world, lantencies were so low that the response to your order submission would land in your front-end before you've had time to lift your finger off the enter key. I now work with web based systems. On days like this, I miss the old ways.