The field of deep learning is commonly concerned with optimizing predictive models using large pre-acquired datasets of densely sampled datapoints or signals. In this work, we demonstrate that the deep learning paradigm can be extended to incorporate a subsampling scheme that is jointly optimized under a desired minimum sample rate. We present Deep Probabilistic Subsampling (DPS), a widely applicable framework for task-adaptive compressed sensing that enables end-toend optimization of an optimal subset of signal samples with a subsequent model that performs a required task. We demonstrate strong performance on reconstruction and classification tasks of a toy dataset, MNIST, and CIFAR10 under stringent subsampling rates in both the pixel and the spatial frequency domain. Due to the task-agnostic nature of the framework, DPS is directly applicable to all real-world domains that benefit from sample rate reduction.
This paper introduces  a novel DPS(Deep Probabilistic Subsampling) framework for the task-adaptive  subsampling case, which attempts to resolve the issue of end-to-end optimization of an optimal subset of signal with jointly learning a sub-Nyquist sampling scheme and a predictive model for downstream tasks. The parameterization is used to simplify the subsampling distribution and ensure an expressive yet tractable distribution.  The new approach contribution is applied to  both reconstruction and classification tasks and demonstrated with a suite of experiments in a toy dataset, MINIST, and COFAR10.
Overall, the paper requires significant improvement. 
1. The approach is not well justified either by theory or practice. There is no experiment clearly shows convincing evidence of the correctness of the proposed approach or its utility compared to existing approaches (Xie & Ermon (2019); Kool et al. (2019); PlÂšotz & Roth (2018) ).
2. The paper never clearly demonstrates the problem they are trying to solve (nor well differentiates it from the compressed sensing problem  or sample selection problem)
   The method is difficult to understand, missing many details and essential explanation, and generally does not support a significant contribution. 
3. The paper is not nicely written or rather easy to follow. The model is not well motivated and the optimization algorithm is also not well described.
4. A theoretical analysis of the convergence of the optimization algorithm could be needed.
5. The paper is imprecise and unpolished and the presentation needs improvement.
**There are so many missing details or questions to answer**
1. What is the Gumbel-max trick? 
2. How to tune the parameters discussed in training details in the experiments?
3. Why to use experience replay for the linear experiments?
4. Are there evaluations on the utility of proposed compared to existing approaches?
5. Does the proposed approach work in real-world problems?
6. Was there any concrete theoretical guarantee to ensure the convergence of the algorithm.
[Post Review after discussion]: The uploaded version has significantly improved over the first submission. It is now acceptable.