TESLA: Task-wise Early Stopping and Loss Aggregation for Dynamic Neural Network Inference
For inference operations in deep neural networks on end devices, it is desirable to deploy a single pre-trained neural network model, which can dynamically scale across a computation range without comprising accuracy. To achieve this goal, Incomplete Dot Product (IDP) has been proposed to use only a subset of terms in dot products during forward propagation. However, there are some limitations, including noticeable performance degradation in operating regions with low computational costs, and essential performance limitations since IDP uses hand-crafted profile coefficients. In this paper, we extend IDP by proposing new training algorithms involving a single profile, which may be trainable or pre-determined, to significantly improve the overall performance, especially in operating regions with low computational costs. Specifically, we propose the Task-wise Early Stopping and Loss Aggregation (TESLA) algorithm, which is showed in our 3-layer multilayer perceptron on MNIST that outperforms the original IDP by 32\% when only 10\% of dot products terms are used and achieves 94.7\% accuracy on average. By introducing trainable profile coefficients, TESLA further improves the accuracy to 95.5\% without specifying coefficients in advance. Besides, TESLA is applied to the VGG-16 model, which achieves 80\% accuracy using only 20\% of dot product terms on CIFAR-10 and also keeps 60\% accuracy using only 30\% of dot product terms on CIFAR-100, but the original IDP performs like a random guess in these two datasets at such low computation costs. Finally, we visualize the learned representations at different dot product percentages by class activation map and show that, by applying TESLA, the learned representations can adapt over a wide range of operation regions.
 This paper presents a modification of a numeric solution: Incomplete Dot Product (IDP), that allows a trained network to be used under different hardware constraints. The IDP method works by incorporating a *coefficient* to each layer (fully connected or convolution), which can be learned as the weights of the model are being optimized. These coefficients can be used to prune subsets of the nodes or filters, when hardware has limited computational capacity. The original IDP method (cited in the paper) is based on iteratively training for higher hardware capacities. This paper improves upon the limitation of the original IDP by allowing the weights of the network be trained concurrently with these coefficients, and authors present a loss function that is linear combination of loss function under original or constrained network setting. They also present results for a *harmonic* combination which was not explained in the paper at all. Overall the paper has very good motivation and significance. However the writing is not very clear and the paper is not self-contained at all. I was not able to understand the significance of early stopping and how this connects with loss aggregation, and how the learning process differs from the original IDP paper, if they also have a scheduled learning setting. Additionally, there were several terms that were unexplained in this paper such as *harmonic* method highlighted in Figure 3. As is, while results are promising, I can*t fully assess that the paper has major contributions.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
TESLA: Task-wise Early Stopping and Loss Aggregation for Dynamic Neural Network Inference
For inference operations in deep neural networks on end devices, it is desirable to deploy a single pre-trained neural network model, which can dynamically scale across a computation range without comprising accuracy. To achieve this goal, Incomplete Dot Product (IDP) has been proposed to use only a subset of terms in dot products during forward propagation. However, there are some limitations, including noticeable performance degradation in operating regions with low computational costs, and essential performance limitations since IDP uses hand-crafted profile coefficients. In this paper, we extend IDP by proposing new training algorithms involving a single profile, which may be trainable or pre-determined, to significantly improve the overall performance, especially in operating regions with low computational costs. Specifically, we propose the Task-wise Early Stopping and Loss Aggregation (TESLA) algorithm, which is showed in our 3-layer multilayer perceptron on MNIST that outperforms the original IDP by 32\% when only 10\% of dot products terms are used and achieves 94.7\% accuracy on average. By introducing trainable profile coefficients, TESLA further improves the accuracy to 95.5\% without specifying coefficients in advance. Besides, TESLA is applied to the VGG-16 model, which achieves 80\% accuracy using only 20\% of dot product terms on CIFAR-10 and also keeps 60\% accuracy using only 30\% of dot product terms on CIFAR-100, but the original IDP performs like a random guess in these two datasets at such low computation costs. Finally, we visualize the learned representations at different dot product percentages by class activation map and show that, by applying TESLA, the learned representations can adapt over a wide range of operation regions.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 This paper presents a modification of a numeric solution: Incomplete Dot Product (IDP), that allows a trained network to be used under different hardware constraints. The IDP method works by incorporating a *coefficient* to each layer (fully connected or convolution), which can be learned as the weights of the model are being optimized. These coefficients can be used to prune subsets of the nodes or filters, when hardware has limited computational capacity. The original IDP method (cited in the paper) is based on iteratively training for higher hardware capacities. This paper improves upon the limitation of the original IDP by allowing the weights of the network be trained concurrently with these coefficients, and authors present a loss function that is linear combination of loss function under original or constrained network setting. They also present results for a *harmonic* combination which was not explained in the paper at all. Overall the paper has very good motivation and significance. However the writing is not very clear and the paper is not self-contained at all. I was not able to understand the significance of early stopping and how this connects with loss aggregation, and how the learning process differs from the original IDP paper, if they also have a scheduled learning setting. Additionally, there were several terms that were unexplained in this paper such as *harmonic* method highlighted in Figure 3. As is, while results are promising, I can*t fully assess that the paper has major contributions.
assistant
 This paper presents a modification of a numeric solution: Incomplete Dot Product (IDP), that allows a trained network to be used under different hardware constraints. The IDP method works by incorporating a *coefficient* to each layer (fully connected or convolution), which can be learned as the weights of the model are being optimized. These coefficients can be used to prune subsets of the nodes or filters, when hardware has limited computational capacity. The original IDP method (cited in the paper) is based on iteratively training for higher hardware capacities. This paper improves upon the limitation of the original IDP by allowing the weights of the network be trained concurrently with these coefficients, and authors present a loss function that is linear combination of loss function under original or constrained network setting. They also present results for a *harmonic* combination which was not explained in the paper at all. Overall the paper has very good motivation and significance. However the writing is not very clear and the paper is not self-contained at all. I was not able to understand the significance of early stopping and how this connects with loss aggregation, and how the learning process differs from the original IDP paper, if they also have a scheduled learning setting. Additionally, there were several terms that were unexplained in this paper such as *harmonic* method highlighted in Figure 3. As is, while results are promising, I can*t fully assess that the paper has major contributions.