Recurrent neural networks (RNN) are powerful time series modeling tools in machine learning. It has been successfully applied in a variety of fields such as natural language processing (Mikolov et al.
The paper attempts to establish the asymptotic accuracy of the "RNN" (but not the RNN models that are well-known in the literature - see the below comments) as a universal functional approximator. It considers a general state space model and uses feedforward neural nets (FNN) to learn the filtering and forecast distributions. Based on the well-known universal approximation property of FNNs, the paper shows that their RNN-based filter can approximate arbitrarily well the optimal filter. 
The paper targets an important problem in statistical learning and offers some interesting insights. However, their so-called RNN-based filter formulation is not anywhere close to the usual RNN models, such as Elman's basic RNN model or LSTM, that are currently known in the literature. Hence the paper's title "RNNs are universal filters" and the way it is presenting are confusing. I think what exactly the paper is about is as follows. They consider a general state space model, then use FNNs to approximate the so-called transition equation and observation equation of that SS model. Then, the resulting RNN-based filter is shown to be able to approximate well the optimal filter of the original SS model. I didn't check the proof carefully but I guess it's intuitively straightforward given the available results from the approximation capacity of FNNs. 
Therefore, I suggest the authors re-write the paper carefully to reflect better the problem that it actually targets. The property of the RNN-based filter is interesting, but using it is, I believe, very difficult from a practical point of view. It's well-known that doing particle filters is computationally expensive, especially in high dimensions, and the RNN-based filter might have millions of parameters! Could the authors please give some comments/discussion about this issue?
Some minor points:
1)  what exactly does "synthesized" mean? 
2) Page 5: "Note that the probability space ... with finite moment..." doesn't read well. What is the moment of probability space?