Self-host analytics for better privacy and accuracy
Piwik is a great project, but it tends not to work well for handling sites with millions of events per day. Your MySQL table starts to bust at the seams pretty quickly.<p>For big sites, you&#x27;ll want that event data in GiB&#x27;s of plain raw logs that you can bulk load into tools like BigQuery or Redshift for analysis.<p>My team has built&#x2F;delivered a SaaS web content analytics platform for the past few years called Parse.ly. We instrument pageview events (like Google Analytics) automatically, and we also instrument time-on-page using heartbeat events. We collect 50 billion monthly page events for over 600 top-traffic sites, and display it all in a real-time dashboard.<p>To emphasize that our customers own the data they send us, we recently launched a Raw Data Pipeline product:<p><a href="http:&#x2F;&#x2F;parse.ly&#x2F;data-pipeline" rel="nofollow">http:&#x2F;&#x2F;parse.ly&#x2F;data-pipeline</a><p>Basically, we host a secure S3 bucket and Kinesis stream for customers, and deliver their raw (enriched) event data there. From there, they typically load it into their own BigQuery or Redshift instance, or they analyze pieces of it directly with Python&#x2F;R&#x2F;Excel&#x2F;etc.<p>Our customers tell us this strikes the right balance among data ownership, query flexibility, and hassle-free infrastructure.