Applying resolution-preserving blocks is a common practice to maximize information preservation in video prediction, yet their high memory consumption greatly limits their application scenarios. We propose CrevNet, a Conditionally Reversible Network that uses reversible architectures to build a bijective two-way autoencoder and its complementary recurrent predictor. Our model enjoys the theoretically guaranteed property of no information loss during the feature extraction, much lower memory consumption and computational efficiency. The lightweight nature of our model enables us to incorporate 3D convolutions without concern of memory bottleneck, enhancing the model's ability to capture both short-term and long-term temporal dependencies. Our proposed approach achieves state-of-the-art results on Moving MNIST, Traffic4cast and KITTI datasets. We further demonstrate the transferability of our self-supervised learning method by exploiting its learnt features for object detection on KITTI. Our competitive results indicate the potential of using CrevNet as a generative pre-training strategy to guide downstream tasks.
This paper introduces Conditionally Reversible Network (CrevNet) that consists of the invertible autoencoder and a reversible predictive module (RPM). The two-way autoencoder is an invertible network that preserves the volume with no information loss while reducing memory consumption by using bijective downsampling. The RPM is a recurrent extension of two-way autoencoder that provides the reversiblity in temporal domain. The experiments on Moving MNIST, Traffic4cast, KITTI, and 2D object detection on KITTI show the improvement compare to other state-of-the-art models. 
The paper is well-written and the contributions are clear. The experiments on diverse tasks and datasets are provided. Especially, the use of pretrained generative model for the object detection task shows the benefit of the model.   
However, it is not clear the necessity of the proposed model and how the memory demand and computation cost are reduced. 
To have a better support for the contributions, I have following suggestions. 
- The benefit or necessity of each module is not clear. The comparisons with 1) only the invertible two-way autoencoder (without Section 2.2 and 2.3)", 2) only RPM (without Section 2.1 and/or 2.3), and 3)two-way autoencoder + reversible predictive model without 3D convolutions (without Section 2.3) are required. 
- Model size/memory comparison is provided only for the MNIST dataset. Since the main benefit of the proposed model is low memory demand and computation cost, I suggest to provide the the comparisons of the memory demand with other models as well.  
1) Other models on on Caltech Pedestrian dataset, Especially, the performance gap on Caltech Pedestrian dataset between the proposed method and CycleGAN is very small. 
2) Some of the compared models on 2D KITTI detection are also lightweighted e.g., SqeeqeDet. The comparison of the model size/memery/test time with these models are necessary. 
---- 
<After rebuttal>
Authors have mostly addressed my concerns, and the contributions are clearer. I adjusted my rating accordingly.