Apache Arrow 1.0
Parquet&#x2F;arrow is a great format for Pandas - fast and nicely compact in terms of file size for those of us who don&#x27;t have the luxury of directly attached NVMe SSDs and where I&#x2F;O bottlenecks are a consideration.<p>Even after issues with its inability to map datetime64 values properly, I was reasonably happy with my design choice.<p>I became less happy on discovering that it&#x27;s very weak as an   interchange format for cross-language work.<p>In my case I wanted to use some existing JVM-based tooling.  This caused huge pain.  The Jvm&#x2F;Java library&#x2F;API is a complete mess, sorry, and if people are complaining about the Python&#x2F;C++ documentation, there&#x27;s basically nothing for the Java library.  
It&#x27;s barely useable and the dependencies are horrific - the whole thing is mingled with hadoop dependencies - even the API itself.<p>And the API is barely above exposing the file-format.  Nothing like &quot;load this parquet file&quot; into some object which you can then query for it&#x27;s contents - you&#x27;re dealing with blocks and sections and other file-format level entities.<p>The other issue is caused its flexibility - for example Panda&#x27;s dataframes are written with what&#x27;s effectively a bunch of &quot;extension metadata&quot; which means it works great for reading and writing pandas from Python but don&#x27;t expect anything to be able to work with the files out-of-the-box in other languages.<p>In the end, the only way I could get reliable reading and writing from the JVM was to only store numeric and string data from the Python side.  Even then it feels flakey - with a bunch of hadoop warnings and deprecation warnings.  I know the JVM has little appreciation in the data science world which is maybe a reason for the sorry state of the Java library.<p>Edit: to be specific, I am talking about my experiences with Arrow&#x2F;Parquet.