DuckDB – An embeddable SQL database like SQLite, but supports Postgres features
I spent a while looking at this today. It's really interesting.<p>It's not based on SQLite at all (except for borrowing the SQLite shell implementation) but it looks very much like SQLite, in particular:<p>- It's designed to work as an embedded library, eliminating the network overhead you usually get when talking to a database<p>- Each database is a single file on disk<p>- It ships as an "amalgamation" build - a single giant C++ file (SQLite is a single giant C file)<p>Impressively, if you run "pip install duckdb" it Just Works - you can then "import duckdb" and start using it, with an interface that looks very similar to the sqlite3 module that ships with Python.<p>The key reason this exists is that it's a column store, with vectorized operations across columns - making it ideal for analytical workloads. This blog entry has some benchmarks that illustrate how well it works in that regard: <a href="https://uwekorn.com/2019/10/19/taking-duckdb-for-a-spin.html" rel="nofollow">https://uwekorn.com/2019/10/19/taking-duckdb-for-a-spin.html</a><p>It's also backed up by some strong computer science. It's by the academic researchers behind MonetDB and includes implementations of a bunch of interesting papers: <a href="https://duckdb.org/docs/why_duckdb#standing-on-the-shoulders-of-giants" rel="nofollow">https://duckdb.org/docs/why_duckdb#standing-on-the-shoulders...</a><p>It's a really interesting piece of software, and unlike many other "new databases" it feels like it fills a very genuine gap in my toolbox.