I was working at a place that had decided that they needed &quot;big data tools&quot; and part of the team was developing a Hadoop data cluster and myriad of other tools (Tableau, Reddis cache etc).<p>I put all the campaign (it was an ad tech company) data that we actually used to alter bidding behaviour and it could fit into a sqlite3 DB that was &lt; 5MB.  Using python the sqlite DB worked like a charm.