I&#x27;ve done more than my fair share of fucking with FITS and ROOT files, HDF5, SQLite, proprietary struct-based things, etc...<p>It&#x27;s easy to get a file format working on one system. It&#x27;s Herculean getting it working on all systems bug free. It&#x27;s nearly impossible to get something to work portably and performant across many systems.<p>As for simplicity, people start wanting metadata and headers and this and that, and before you know it you need HDF5 or ROOT again and it&#x27;s no longer simple. Maybe if you&#x27;re lucky you can stick with something that looks like FITS. If it&#x27;s tabular, SQLite still can&#x27;t be beat. Maybe Parquet would work fine too.<p>I&#x27;d vehemently oppose anyone in the projects I work on from trying to standardize on a new in-house format. I&#x27;d maybe be okay if they were just building on top of MessagePack or Cap&#x27;n Proto&#x2F;thrift etc... but nearly every disadvantage the OP references about HDF5 will undoubtedly be in anything they cook up themselves. For example, a &quot;simpler format&quot; that works well on distributed architectures, well... now you&#x27;re going to go back to the single implementation problem.