Show HN: ION, a JSON alternative – Versatile, compact, fast, binary data format
I&#x27;m looking at: <a href="http:&#x2F;&#x2F;tutorials.jenkov.com&#x2F;iap&#x2F;ion-vs-other-formats.html" rel="nofollow">http:&#x2F;&#x2F;tutorials.jenkov.com&#x2F;iap&#x2F;ion-vs-other-formats.html</a><p>As the author of Protobuf v2 (the version that was open sourced by Google), I object to some of the &quot;no&quot;s in the protobuf column.<p>(Note: I no longer work on Protobuf, and I did not invent the format. I do work on and did invent Cap&#x27;n Proto.)<p>&gt; Protobuf apparently isn&#x27;t great at encoding raw bytes either (according to their own website).<p>Protobuf can handle raw bytes just fine, using the &quot;bytes&quot; type. There is no special encoding done on bytes; parsing and encoding is done by memcpy(). I&#x27;m curious to know what part of the web site you interpret as saying otherwise. It&#x27;s entirely possible that the web site contains confusing language, but a citation would have been a good idea here.<p>&gt; Schema &#x2F; Class Id
&gt; Self describing<p>The Protobuf libraries have extensive support for manipulating dynamic schemas and transmitting schemas over the wire. See the &quot;Descriptor&quot; and &quot;DynamicMessage&quot; APIs. This is mentioned on the web site:<p><a href="https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers&#x2F;docs&#x2F;techniques#self-description" rel="nofollow">https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers&#x2F;docs&#x2F;techniqu...</a><p>&gt; Even if these compact objects do not contain any property names, they are still self describing enough that you can see where fields start and end, plus their data type, without an external schema. You cannot do that with Protobuf (as far as we know).<p>You absolutely can do that with Protobuf. This is what the &quot;protoc --decode_raw&quot; flag does, and it should be clear enough from reading the encoding.<p><a href="https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers&#x2F;docs&#x2F;encoding" rel="nofollow">https:&#x2F;&#x2F;developers.google.com&#x2F;protocol-buffers&#x2F;docs&#x2F;encoding</a><p>&gt; Cyclic references<p>While it&#x27;s true that Protobuf doesn&#x27;t support these, I hope you&#x27;ve considered the denial-of-service vulnerabilities they tend to create if the receiver is not expecting them. Please ensure that cyclic references are only allowed in cases where the app opted into it.<p>Relatedly, overlapping references &#x2F; backreferences (&quot;Copy&quot; in your table) potentially leads to an amplification attack where a small message on the wire turns out to be much, much larger when traversed. If applications cannot defend themselves from huge payloads by setting a message size limit, then you&#x27;ll need to give them some other way.<p>&gt; All of the formats (except perhaps Protobuf) supports arbitrary hierarchical navigation of the encoded data, without first converting it to objects.<p>Protobuf supports this, and in fact should be an unqualified &quot;Yes&quot; rather than &quot;Yes(*)&quot; like the others. Protobuf encoding is very similar to ION&#x27;s. Sub-messages are length-delimited, which seems to be exactly the advantage you&#x27;re claiming that ION has.<p>Note that none of these formats support random access in the way that Cap&#x27;n Proto does.<p>In summary, I believe Protobuf deserves a &quot;yes&quot; in: &quot;Raw bytes&quot;, &quot;Good at raw bytes&quot;, &quot;Schema &#x2F; Class Id&quot;, &quot;Arbitrary hierarchical navigation&quot;, and &quot;Self describing&quot;.