DuckDB – An embeddable SQL database like SQLite, but supports Postgres features
I spent a while looking at this today. It's really interesting.It's not based on SQLite at all (except for borrowing the SQLite shell implementation) but it looks very much like SQLite, in particular:- It's designed to work as an embedded library, eliminating the network overhead you usually get when talking to a database- Each database is a single file on disk- It ships as an "amalgamation" build - a single giant C++ file (SQLite is a single giant C file)Impressively, if you run "pip install duckdb" it Just Works - you can then "import duckdb" and start using it, with an interface that looks very similar to the sqlite3 module that ships with Python.The key reason this exists is that it's a column store, with vectorized operations across columns - making it ideal for analytical workloads. This blog entry has some benchmarks that illustrate how well it works in that regard: https://uwekorn.com/2019/10/19/taking-duckdb-for-a-spin.htmlIt's also backed up by some strong computer science. It's by the academic researchers behind MonetDB and includes implementations of a bunch of interesting papers: https://duckdb.org/docs/why_duckdb#standing-on-the-shoulders...It's a really interesting piece of software, and unlike many other "new databases" it feels like it fills a very genuine gap in my toolbox.