Inserting 130M SQLite rows per minute from a scripting language
Personally I'm more interested in <i>indexing</i> a boatload of data as fast as possible, with modest resource requirements. Had a couple of cases where I'd like to do that on a mid-tier laptop, with no particular success so far. I have to guess, but it seems that ‘data scientists’ either buy big fat boxes with tons of ram and cpu, or offload everything to big fat boxes in datacenters, or twiddle thumbs for a quite while. You'd think that by now writing indexes on all columns at the top sequential drive speed would be a solved problem from any ‘Learn data science in two days’ tutorial.