Why DNSFilter replaced InfluxDB with TimescaleDB
Influx founder and CTO here. I figure I should comment because it seems this post is a bit dated on information. That being said, I think this is one of those cases where based on where InfluxDB was at the time, they simply picked the wrong tool for the job. We were looking at use cases with hundreds of thousands of series. It just wasn&#x27;t designed for what they were trying to do (for whatever version they were running at the time). We have much higher aspirations now, but I&#x27;ll get to that in a bit when I talk to each point raised in the post.<p>As I mentioned, it would be helpful to know which version of InfluxDB was tested against, but I&#x27;ll try to cover each of the points.<p>1 - Cardinality. High cardinality has long been a known problem for InfluxDB. We started work on this almost 15 months ago to create a new feature called TSI or Time Series Index. It was significantly harder for us to develop than we thought it would be. It can be enabled on version 1.5, which was released yesterday. We&#x27;ll have a detailed whitepaper on how it works and what techniques we used coming soon. It shows significantly lower memory usage for high cardinality write workloads. The other side of this is query, which I&#x27;ll talk about in a bit.<p>2 - Rollup tables is a known issue for us. Continuous queries were originally supposed to do this, but the problem is that it falls over at scale. There are some other weird things about how it works where it won&#x27;t pick up lagged data collection. It&#x27;s a weak spot for us, Prometheus, OpenTSDB and a bunch of other solutions. From what I can tell, Timescale doesn&#x27;t offer this feature either. See <a href="https:&#x2F;&#x2F;github.com&#x2F;timescale&#x2F;timescaledb&#x2F;issues&#x2F;350" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;timescale&#x2F;timescaledb&#x2F;issues&#x2F;350</a>. It sounds like DNSFilter implemented this themselves using their Kafka ingest pipeline and computing them and inserting into Timescale tables (which for now is exactly what we recommend with InfluxDB). If I&#x27;m guessing wrong, I&#x27;d love to hear the detail about how this works in your solution.<p>3 - Ingestion performance vs. Query performance. It sounds like query load causes ingestion performance to degrade. I really don&#x27;t know without more information. In the post Mike said they were working with InfluxDB in 2015 and 2016 and didn&#x27;t continue upgrading. So they could have been running 0.11.0, a version that is now over two years old. We&#x27;ve made significant improvements since then. However, I will say that our current query engine is a known weak spot. This is why we&#x27;re investing heavily into IFQL (our new language and engine). The new language also addresses the InfluxQL doesn&#x27;t actually operate like SQL, which is something he mentions at the top of the post. I agree that can cause frustration, which is why we&#x27;re designing our new language around this use case. I think the functional paradigm makes more sense than SQL for time series. Slides from a recent talk I gave about the engine:<p>Slides:
<a href="https:&#x2F;&#x2F;speakerdeck.com&#x2F;pauldix&#x2F;ifql-and-the-future-of-influxdata" rel="nofollow">https:&#x2F;&#x2F;speakerdeck.com&#x2F;pauldix&#x2F;ifql-and-the-future-of-influ...</a>
Video:
<a href="https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=QuCIhTL2lQY&amp;list=PLYt2jfZorkDqmNVloKpJGp-47KVeKcUPb" rel="nofollow">https:&#x2F;&#x2F;www.youtube.com&#x2F;watch?v=QuCIhTL2lQY&amp;list=PLYt2jfZork...</a><p>4 - Resource utilization. It uses RAM, particularly on high cardinality workloads. Yes, we know this and have known it for a long time. Running tests on 1.5 with TSI enabled, the RAM utilization is significantly lower. However, TSI is only one side of the issue. The other side is querying high cardinality data. Our current query engine will eat many resources trying to do this. It&#x27;s something we&#x27;re addressing with the IFQL engine. You can actually use the new engine as a separate process against the InfluxDB 1.5 release. However, it&#x27;s still under heavy work and we haven&#x27;t begun the performance optimizations.<p>I&#x27;m confused how Mike talks about about their query processor and then immediately dives into Kafka, which is only relevant in the ingestion pipeline. In fact, our recommended configuration is to route writes through Kafka before sending to InfluxDB if you&#x27;re operating at scale. It&#x27;s how we&#x27;re designing 2.0 from day 1 and that work has already started. In an analytics pipeline, you want to separate your write pipeline from production query&#x2F;storage.<p>1 - Ease of change: schema changes in InfluxDB aren&#x27;t easy. Postgres supports alter table commands. How do these work on large tables? Has Timescale (or Postgres) solved the problem of these kinds of operations taking the DB down for a while? I think that&#x27;s one of the strengths of having Kafka in their ingestion pipeline to protect against it (or other DB issues). We still need to do work here and I know it&#x27;s a weak spot and it&#x27;s honestly just a very hard problem (for how data is stored on disk for us). In the future, at the very least we&#x27;ll enable users to kick a schema change off and have it run in the background while keeping the production infrastructure up. Depending on the change it might be a long running background task that requires rewriting many things.<p>2 - Performance - Impossible to address without knowing their data, their queries, and really testing it on a new version. We&#x27;re optimizing all the time. I guarantee you that the query performance you see in InfluxDB today won&#x27;t be anything close to as good as what it&#x27;ll be a year from now.<p>3 - No missing data - I don&#x27;t know of any issue in the current release (or many previous releases) of Influx that just drops writes. This might be something to do with Continuous Queries since it sounded like it was about rollups. It&#x27;s a known issue with that feature and it&#x27;s being completely reworked in 2.0 to work in a guaranteed way, at scale. It&#x27;s a long running problem and one that is hard to get right. Again, it sounds like Mike and team implemented it themselves through their ingestion pipeline and then attributed the gain to Timescale. You can do the same thing with InfluxDB and it&#x27;s our recommended architecture. We tell customers to do their downsampling<p>I&#x27;m not sure what is going on with deletes. You can do deletes in Influx and you can delete by a time range, which would give you a specific point. That being said we&#x27;ve done significant work in the last three releases to improve how deletes work and particularly how they work for deleting large amounts of data.<p>For loads, I&#x27;d really like to see a comparison against the 1.5 release with TSI on compared to Timescale. A release that&#x27;s 1.5 to 2 years old with InfluxDB is completely different that what is current, as many people that have been watching the project over the last 4 years has seen.<p>Finally, Mike does make a case at the end for Timescale just being SQL and Postgres so they found it easy to use because it&#x27;s familiar. For better or worse, we&#x27;re making the bet that SQL isn&#x27;t the last only true API for working with time series data. Some developers will prefer SQL and some will prefer other approaches. For InfluxDB, we&#x27;re making a bet that other approaches will be better for developer productivity.