Topic modeling of text documents is one of the most important tasks in representation learning. In this work, we propose iTM-VAE, which is a Bayesian nonparametric (BNP) topic model with variational auto-encoders. On one hand, as a BNP topic model, iTM-VAE potentially has infinite topics and can adapt the topic number to data automatically. On the other hand, different with the other BNP topic models, the inference of iTM-VAE is modeled by neural networks, which has rich representation capacity and can be computed in a simple feed-forward manner. Two variants of iTM-VAE are also proposed in this paper, where iTM-VAE-Prod models the generative process in products-of-experts fashion for better performance and iTM-VAE-G places a prior over the concentration parameter such that the model can adapt a suitable concentration parameter to data automatically. Experimental results on 20News and Reuters RCV1-V2 datasets show that the proposed models outperform the state-of-the-arts in terms of perplexity, topic coherence and document retrieval tasks. Moreover, the ability of adjusting the concentration parameter to data is also confirmed by experiments.
"topic modeling of text documents one of most important tasks"
Does this claim have any backing?
"inference of HDP is more complicated and not easy to be applied to new models"  Really an artifact of the misguided nature of earlier work. The posterior for the $\vec\pi$ of a elements of DP or HDP can be made a Dirichlet, made finite by keeping a "remainder" term and appropriate augmentation.  Hughes, Kim and Sudderth (2015) have avoided stick-breaking and CRPs altogether, as have others in earlier work. Extensive models building on simple HDP doing all sorts of things have been developed.
Variational stick-breaking methods never seemed to have worked well.  I suspect you could achieve better results by replacing them as well, but you would have to replace the tree of betas and extend your Kumaraswamy distribution, so it may not work.  Anyway, perhaps an avenue for future work.
"infinite topic models" I've always taken the view that the use of the word "infinite" in machine learning is a kind of NIPSian machismo. In HDP-LDA at least, the major benefit in model performance comes from fitting what you call $\vec\pi$, which is uniform in vanilla LDA, and note that the number of topics "found" by a HDP-LDA sampler can be made to vary quite widely by varying what you call $\alpha$, so any statement about the "right" number of topics is questionable.  So the claim in 3rd paragraph of Section 2, "superior" and "self-determined topic number" I'd say are misguided.  Plenty of experimental work to support this.
In Related Work, you seem to only mention HDP for non-parametric topic models.  More work exists, for instance using Pitman-Yor distributions for modelling words and using Gibbs samplers that are efficient and don't rely on the memory hungry HCRP.
Good to see a prior is placed on the concentration parameter.  Very important and not well done in the community, usually.  
ADDED:  Originally done by Teh et al for HDP-LDA, and subsequently done
by several, including Kim et al 2016.   Others stress the importance of this.  You need to
cite at least Teh et al. in 5.4 to show this isn't new and the importance is well known.
The Prod version is a very nice idea.  Great results.  This looks original, but I'm not expert enough in the huge masses of new deep neural network research popping up.
You've upped the standard a bit by doing good experimental work.  Oftentimes this is done poorly and one is left wondering.  A lot of effort went into this.
ADDED:   usually like to see more data sets experimented with
What code is used for HDP-LDA?  Teh's original Matlab HCRP sampler does pretty well because at least he samples hyperparameters and can scale to 100k documents (yes, I tried). The comparison with LDA makes me suspicious. For instance, on 20News, a good non-parametric LDA will find well over 400 topics and roundly beat LDA on just 50 or 200.  If reporting LDA, or HDP-LDA, it should be standard to do hyperparameter fitting and you need to mention what you did as this makes a big difference.
ADDED:   20News results still poor for HPD, but its probably the implementation used ... their
        online variational algorithm only has advantages for large data sets 
Pros:   
* interesting new prod model with good results
* alternative "deep" approach to a HDL-LDA model
* good(-ish) experimental work
Cons:
* could do with a competitive non-parametric LDA implementation
ADDED:   good review responses generally