One SQL to Rule Them All: Management of Streams and Tables
A really interesting proposal to unify stream processing and relational querying and to extend SQL with a semantics over time-varying relations.<p>In particular, the authors (from the Apache Beam/Flink/Calcite projects) stress on the need to:<p>* smooth the frontier between streams and tables by using a single concept of time-varying relations;<p>* make explicit the difference between the event and processing times, using monotonic watermarks to tie the two;<p>* control when these continuously varying relations are materialized, avoiding unnecessary work and producing only timely outcomes.<p>These ideas are not new per se, but there are here pushed further and nicely combined.<p>However, I'm wondering if SQL is the right tool for the task. For instance, the listing 2 seems complex when compared to the query expressed in plain English. In particular, I disagree that "anyone who understands enough SQL to solve a problem in a non-streaming context still has the knowledge required to solve the problem in a streaming context as well.<p>A must read paper by who is keen on stream processing, Spark, Flink, Beam, Kafka Stream ...