We propose a generic confidence-based approximation that can be plugged in and simplify the auto-regressive generation process with a proved convergence. We first assume that the priors of future samples can be generated in an independently and identically distributed (i.i.d.) manner using an efficient predictor. Given the past samples and future priors, the mother AR model can post-process the priors while the accompanied confidence predictor decides whether the current sample needs a resampling or not. Thanks to the i.i.d. assumption, the post-processing can update each sample in a parallel way, which remarkably accelerates the mother model. Our experiments on different data domains including sequences and images show that the proposed method can successfully capture the complex structures of the data and generate the meaningful future samples with lower computational cost while preserving the sequential relationship of the data.
The authors consider the problem of sampling time series.
To solve the problem they propose a method that is based on the autoregression model. The novelty here lies in the proposed sampling methods: we start with a sampling of a prior and then try to generate data according to the restored distribution. We learn two functions: signal recovery and confidence prediction.
The main hyperparameter of the algorithm $\varepsilon$ identifies how much samples we accept.
The distinctive feature of the algorithm is speed-up for the sample generation process.
Weak reject
There are a significant number of works on video generation, see e.g. [1, 2], references therein and articles that cite these two articles. The problem setting seems pretty similar. It seems like a good idea to compare to these methods (and it seems that video generation is a very resource-demanding procedure, and they don't use parallel applications similar to proposed in the paper. What is the reason?) Most of the approaches use only one frame to generate video, but it seems that LSTM in these methods will benefit from using of multiple frames as input (and will be able to transfer information in autoregression manner by transferring all they need in a hidden state).
The article, in my opinion, will benefit from comparison to these approaches or at least by using some benchmarks from these works to demonstrate feasibility of the considered approach, also it seems that these works are good for demonstration of parallelization capabilities (as in many cases the same idea applies).
Not minor comments:
1. In Figure 2 (a) it is not clear how the data and prediction were generated. According to the procedure in Figure 1 and text we use the same input for all approaches. However solid lines for different epsilons are different.
2. The effect of the dependence of recovery of quality for Figure 2 (b) is not explained and is controversial: we get the smallest error for intermediate acceptance ratio, however, there is also a decrease of error if we further increase the gauge threshold (btw the term gauge threshold is new to machine learning community, consider replacement of it)
3. More simpler examples will benefit the paper, as we'll be able to know more fundamental properties of the proposed approach. 
Minor technical comments:
1. s. 3.1. predictor predicts
commas in equation (8)
2. Figure 2: no axis labels for the left plot, use for label "acceptance ratio" red color font & for label "L1 error" blue color font
3. Table 1 bracket after l_1 is missing
4. Maybe $\sigma$ is not the best designation of confidence, as it can be confused with the variance
5. Figure 1: some indexes should be not $x_{i + 2}$, but $x_{i + j}$, $x_{i + M}$. Also, some ">" before \epsilon should be "<"
6. "a auto-encoder architecture" -> 
"an auto-encoder architecture"
[1] J. He et al. Probabilistic Video Generation using Holistic Attribute Control. 2018. ECCV
[2] E. Denton, R.Fergus. Stochastic Video Generation with a Learned Prior. 2018.