TileDB 2.0 and the Future of Data Science
I don&#x27;t know if the creators of TileDB visit HN, but if they do - it would be nice to see a code example that showcases what makes TileDB unique. Right now, I don&#x27;t see any benefit versus serializing a data frame in hdf5 and storing it in Redis or cloud blob storage.<p>(Reading <a href="https:&#x2F;&#x2F;tiledb.com&#x2F;developer" rel="nofollow">https:&#x2F;&#x2F;tiledb.com&#x2F;developer</a>, the Github, or the blog - none of it shows me anything amazing)<p>Edit: browsing around the site, I finally found out what tileDB offers, and I&#x27;m still confused (too much fluff language) To save people time:<p><a href="https:&#x2F;&#x2F;docs.tiledb.com&#x2F;main&#x2F;handling-dataframes" rel="nofollow">https:&#x2F;&#x2F;docs.tiledb.com&#x2F;main&#x2F;handling-dataframes</a><p>Multi-column slicing: By defining any subset of the columns as dimensions and due to 
TileDB&#x27;s tiling flexibility, you can increase the pruning effectiveness of multi-column slicing, thus leading to better overall read performance. Essentially, a TileDB array acts as a primary multi-dimensional index on the columns selected as the array dimensions.<p>Data updates and versioning: TileDB offers rapid, parallel, cloud-optimized updates. All the update logic is pushed into the storage engine and is completely transparent to the user. TileDB also exposes useful time traveling functionality, such as reading arrays at time snapshots, effectively implementing data versioning built into a single embeddable library.<p>Partitioning: TileDB enables balanced partitioning, without limiting each partition to single column values. Moreover, we will soon expose API functions for dynamically selecting different partitioning schemes (e.g., on different subsets of columns with different orders), without the need for reorganizing&#x2F;rewriting the array.<p>Sorting: All sorting is taken care of by TileDB internally with multi-threading.