Launch HN: Opstrace (YC S19) – open-source Datadog
One pain point with Prometheus is that is has relatively weak support for quantiles, histograms, and sets[1]:<p>- Histograms require manually specifying the distribution of your data, which is time-consuming, lossy, and can introduce significant error bands around your quantile estimates.<p>- Quantiles calculated via the Prometheus &quot;summary&quot; feature are specific to a given host, and not aggregatable, which is almost never what you want (you normally want to see e.g. the 95th percentile value of request latency for all servers of a given type, or all servers within a region). Quantiles can be calculated from histograms instead, but that requires a well-specified histogram and can be expensive at query time.<p>- As far as I know, Prometheus doesn&#x27;t have any explicit support for unique sets. You can compute this at query time, but persisting and then querying high-cardinality data in this way is expensive.<p>Understanding the distribution of your data (rather than just averages) is arguably the most important feature you want from a monitoring dashboard, so the weak support for quantiles is very limiting.<p>Veneur[2] addresses these use-cases for applications that use DogStatsD[3] by using clever data structures for approximate histograms[4] and approximate sets[5], but I believe its integration with Prometheus is limited and currently only one-way - there is a CLI app to poll Prometheus metrics and push them into Veneur[6], but there&#x27;s no output sink for Veneur to write to Prometheus (or expose metrics for a Prometheus instance to poll), and you aren&#x27;t able to use the approximate histogram or approximate set datatypes if you go that route, because they can&#x27;t be expressed as Prometheus metrics.<p>It would be extremely useful to have something similar for Prometheus, either by integrating with Veneur or implementing those data structures as an extension to Prometheus.<p>[1] <a href="https:&#x2F;&#x2F;prometheus.io&#x2F;docs&#x2F;practices&#x2F;histograms&#x2F;" rel="nofollow">https:&#x2F;&#x2F;prometheus.io&#x2F;docs&#x2F;practices&#x2F;histograms&#x2F;</a><p>[2] <a href="https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur</a><p>[3] <a href="https:&#x2F;&#x2F;docs.datadoghq.com&#x2F;developers&#x2F;dogstatsd&#x2F;" rel="nofollow">https:&#x2F;&#x2F;docs.datadoghq.com&#x2F;developers&#x2F;dogstatsd&#x2F;</a><p>[4] <a href="https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur#approximate-histograms" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur#approximate-histograms</a><p>[5] <a href="https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur#approximate-sets" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur#approximate-sets</a><p>[6] <a href="https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur&#x2F;tree&#x2F;master&#x2F;cmd&#x2F;veneur-prometheus&#x2F;" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;stripe&#x2F;veneur&#x2F;tree&#x2F;master&#x2F;cmd&#x2F;veneur-prom...</a>