Data have often to be moved between servers and clients during the inference phase. This is the case, for instance, when large amounts of data are stored on a public storage server without the possibility for the users to directly execute code and, hence, apply machine learning models. Depending on the available bandwidth, this data transfer can become a major bottleneck. We propose a simple yet effective framework that allows to select certain parts of the input data needed for the subsequent application of a given neural network. Both the associated selection masks as well as the neural network are trained simultaneously such that a good model performance is achieved while, at the same time, only a minimal amount of data is selected. During the inference phase, only the parts selected by the masks have to be transferred between the server and the client. Our experiments indicate that it is often possible to significantly reduce the amount of data needed to be transferred without affecting the model performance much.
The paper addresses the problem of high-cost transfer between server and user for machine learning applications. The method proposes to augment inputs with channel/spatial masks that are trained via the Gumbel-softmax trick together with the model's weights trained/finetuned to account for the loss of information. These learned masks are then applied to the image before sending it to a server with where inference takes place to reduce file transfer costs. In the experimental study, the paper shows that on computer vision tasks inputs can be reduced with relatively little drop in accuracy and analyses how hyperparameters of the model affect its performance. The framework is also adapted to the downstream task-guided choice of compression techniques, e.g. which compression quality to choose for JPEG that will be an optimal trade-off in terms of downstream task quality vs data transfer cost.
Overall, this paper could be a valid algorithmic contribution, however, I have concerns about how this method fares with others, resolving the following issues in the author response will likely increase the score.
- A discussion on the connection/comparison with representation learning and dimensionality reduction (VAEs, quantization, etc) would help improve the exposition of the paper and help define how and when the suggested method is more appropriate to use, as well as citations to the other lines of work in reducing the model size (knowledge distillation [1], tensor decomposition approaches [2], adaptive computation time techniques [3], etc).
- I would like to see an experiment that compares other works mentioned as related work. [4] has been mentioned as one of the nontrivial compression methods for image data, how does the proposed method compare to it?
- Another experiment comparing a method from representation learning, e.g. a VAE trained with the embedding size corresponding to some optimal Q value in this work would be helpful.
- Please include reference accuracy values for the dataset/NN pairs used in Table 1.
- In Section 4.5, the paper states that large lambda values correspond to blue and red lines, however in the corresponding figure large lambda values correspond to blue and orange lines which exhibit different behaviors, could you please clarify this?
References
[1] Hinton, Geoffrey, Oriol Vinyals, and Jeff Dean. "Distilling the knowledge in a neural network."
[2] Novikov, Alexander, et al. "Tensorizing neural networks." Advances in neural information processing systems. 2015.
[3] Figurnov, Michael, et al. "Spatially adaptive computation time for residual networks." Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2017.
[4] Jiang, Feng, et al. "An end-to-end compression framework based on convolutional neural networks." IEEE Transactions on Circuits and Systems for Video Technology 28.10 (2017): 3007-3018.