Loading half a billion rows into MySQL
Assuming your event data is immutable (i.e. no UPDATEs, just INSERTs), you'd probably have fewer headaches long-term if you just dumped the database to flatfiles, stored in HDFS and queried using Hive (which has MySQLish query syntax anyway). This architecture will take you to billions of rows quite happily.<p>This is the architecture we use for eventstream analysis at SnowPlow (<a href="https://github.com/snowplow/snowplow" rel="nofollow">https://github.com/snowplow/snowplow</a>).