Ask HN: Good tech talks on how analytics systems are implemented?
My go-to v0 solution is JSON (simple, with no nested objects or lists) written to s3 (partitioned by date, see Hive date partitioning) and AWS Athena (serverless Presto) to do SQL queries on those JSONs. You can build the system in less than an hour, you don&#x27;t have to manage any VMs and it&#x27;s relatively easy to extend to a more serious solution (e.g. if you need major scale or Spark-like analytic jobs).<p>Relational databases like some are suggesting are fine, but you have to manage them unlike s3 + Athena and it tends to make you design around relational database concepts, which can make it difficult to migrate to a full-blown analytics solution that often abandons relational guarantees.<p>This solution also lets you be flexible in your raw data schema unlike relational databases where you have to have well defined schema or hacks like saving the raw JSON as a string.<p>When you need to evolve your data schema (you will as you learn what things you want to measure), a relational database requires you to be thoughtful about how to do this (e.g. you can&#x27;t have your data producer writing a new schema before the table has been changed). Often this requires you to add some sort of queue between data producers and database so that you can make changes to the table without stopping the data producers. With s3 + Athena, you can just upgrade your data producer, it will start saving the new format to s3 and then you upgrade your Athena table definition whenever you want to start querying the new data (because in relational databases, the schema defines how data is stored, but in s3+Athena world, the schema just tells the SQL engine how to read whatever data exists on s3).