We propose a new method Continual Density Ratio Estimation (CDRE), which can estimate density ratios between a target distribution of real samples and a distribution of samples generated by a model while the model is changing over time and the data of the target distribution is not available after a certain time point. This method perfectly fits the setting of continual learning, in which one model is supposed to learn different tasks sequentially and the most crucial restriction is that model has none or very limited access to the data of all learned tasks. Through CDRE, we can evaluate generative models in continual learning using f -divergences. To the best of our knowledge, there is no existing method that can evaluate generative models under the setting of continual learning without storing real samples from the target distribution.
This submission proposes a continual density ratio estimation method and suggested its application to evaluate a continually learned generative model without requiring the previous training data. The basis of the continual density estimation is based on a recursive relationship between the density ratio at step t and that at t - 1. Continually estimating the density ratio without storing the raw data is an interesting topic and could be useful for continual learning. To my knowledge, I have not seen this in earlier publications.
However, I give reject to this paper because of the following reason: 
The writing of this paper is not easy to follow. 
- The beginning of section 3 (CDRE in continual learning), I found it difficult to understand why the model q needs to be updated (indexed by t) while p(x) is not dependent on t. As far as I know, under the continual learning setting the data distribution p(x) is also conditioned on t. I interpret it as a general introduction on how density ratio could be estimated continually.
- The Lagrange multiplier and the bias / variance statements need elaboration, I don't understand how it is affecting the bias and variance.
- In the second part of section 3, the continual learning setting is introduced (in equation 11), however, it is no longer reasonable to use the symbol r_t in equation 12 which was initially defined in equation 5. 
- A loss for continual VAE is proposed in seciton Feature generation for CDRE, however, the p(x) is again independent of t. And I'm also suspicious that equation 13 is the correct way of adjusting VAE's objective with VCL. In VCL, the KL divergence is on the parameter distribution, which could help prevent forgetting, however, here the KL is between VAE's approximate posteriors, which alone is not sufficient for keeping the information of previous tasks.
- There's lack of analysis / interpretation of results for section 4.1, e.g. what is the motivation of the experiments and what is the conclusion.
- Through out section 4.2 - 4.3, it is not explained what is the source of variance in the experiment results.