PCNN: Environment Adaptive Model Without Finetuning
Convolutional Neural Networks (CNNs) have achieved tremendous success for many computer vision tasks, which shows a promising perspective of deploying CNNs on mobile platforms. An obstacle to this promising perspective is the tension between intensive resource consumption of CNNs and limited resource budget on mobile platforms. Existing works generally utilize a simpler architecture with lower accuracy for a higher energy-efficiency, 	extit{i.e.}, trading accuracy for resource consumption. An emerging opportunity to both increasing accuracy and decreasing resource consumption is 	extbf{class skew}, 	extit{i.e.}, the strong temporal and spatial locality of the appearance of classes. However, it is challenging to efficiently utilize the class skew due to both the frequent switches and the huge number of class skews. Existing works use transfer learning to adapt the model towards the class skew during runtime, which consumes resource intensively. In this paper, we propose 	extbf{probability layer}, an 	extit{easily-implemented and highly flexible add-on module} to adapt the model efficiently during runtime 	extit{without any fine-tuning} and achieving an 	extit{equivalent or better} performance than transfer learning. Further, both 	extit{increasing accuracy} and 	extit{decreasing resource consumption} can be achieved during runtime through the combination of probability layer and pruning methods.
 The paper proposes a simple idea to calibrate probabilities outputted by a CNN model to adapt easily to environments where class distributions change with space and time (and are often skewed). The paper shows that such a simple approach is sufficient to get good accuracies without requiring any costly retraining or transfer learning. Thereby proving to give benefits in terms of resource consumption and at the same time giving better results than the state of the art. However, A] The proposed calibration doesn*t take any CNN specific details into consideration, rather it is a general calibration method which was also proposed in Saerens et. al, 2002 (cited in the paper). It is unclear why the paper specifically talks about CNN. B] The proposed Class Skew Detector is a simple method. Change-point detection is a well-studied area. The paper lacks a literature review in this area and a reasoning of why the proposed approach is preferred. Also, an independent analysis of how the class skew detector behaves in the face of rapidly changing class skews versus slow changing class skews is warranted here. Particularly, given that the paper proposes to use this approaches in mobile which may work in both rapid and slow changing class skews. C] The Class Skew Detector is dependent on the base model. Thus, it is also likely that the empirical distribution estimated is biased and yet the final accuracies reported are much higher than the base model accuracies. There is something interesting happening here. An analysis of the robustness of the proposed approach in the face of noisy class skew detection could potentially make this paper a stronger work. D] The analysis in the paper has largely focused on pre-trained models. However, another analysis that could have been useful here is, varying the quality of the classifier (e.g. classifier trained on skewed training data vs. balanced training data) and measuring how the quality of the classifier correlates with the final performance. Maybe even attempt to answer the question "which classifiers are likely to work with this approach?" In fact, this analysis can be either done in a general context of any classifier or just CNN*s and identifying whether certain properties of CNN help in getting better performance. The paper lacks novelty and at the same time, it is not quite compensating that with a detailed analysis of the work. The problem is interesting and I like the work because the approach is simple and the results look good. I think with a stronger focus on more detailed analysis, this can be a good submission to an applied conference like MobiCom etc. By the way, the paper is riddled with several spelling errors - "filed" -> "field", page 1, second paragraph, last line "complimentary" -> "complementary", page 2, section 2, paragraph 1, last line "epoches" -> "epochs", page 2, section 2, transfer learning, second paragraph, second last line "CNNs does not use" -> "CNNs do not use", page 3, section 3, intuition, first paragraph, first line "formular" -> "formula", page 4, above equation 4 Equation 4 has a typo in the denominator, P_t(i) should be P_t(j), same with Equation 5 "obstained" -> "obtained", page 7, second paragraph, first line "adaptation" is almost everywhere spelled as "adaption"
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
PCNN: Environment Adaptive Model Without Finetuning
Convolutional Neural Networks (CNNs) have achieved tremendous success for many computer vision tasks, which shows a promising perspective of deploying CNNs on mobile platforms. An obstacle to this promising perspective is the tension between intensive resource consumption of CNNs and limited resource budget on mobile platforms. Existing works generally utilize a simpler architecture with lower accuracy for a higher energy-efficiency, 	extit{i.e.}, trading accuracy for resource consumption. An emerging opportunity to both increasing accuracy and decreasing resource consumption is 	extbf{class skew}, 	extit{i.e.}, the strong temporal and spatial locality of the appearance of classes. However, it is challenging to efficiently utilize the class skew due to both the frequent switches and the huge number of class skews. Existing works use transfer learning to adapt the model towards the class skew during runtime, which consumes resource intensively. In this paper, we propose 	extbf{probability layer}, an 	extit{easily-implemented and highly flexible add-on module} to adapt the model efficiently during runtime 	extit{without any fine-tuning} and achieving an 	extit{equivalent or better} performance than transfer learning. Further, both 	extit{increasing accuracy} and 	extit{decreasing resource consumption} can be achieved during runtime through the combination of probability layer and pruning methods.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper proposes a simple idea to calibrate probabilities outputted by a CNN model to adapt easily to environments where class distributions change with space and time (and are often skewed). The paper shows that such a simple approach is sufficient to get good accuracies without requiring any costly retraining or transfer learning. Thereby proving to give benefits in terms of resource consumption and at the same time giving better results than the state of the art. However, A] The proposed calibration doesn*t take any CNN specific details into consideration, rather it is a general calibration method which was also proposed in Saerens et. al, 2002 (cited in the paper). It is unclear why the paper specifically talks about CNN. B] The proposed Class Skew Detector is a simple method. Change-point detection is a well-studied area. The paper lacks a literature review in this area and a reasoning of why the proposed approach is preferred. Also, an independent analysis of how the class skew detector behaves in the face of rapidly changing class skews versus slow changing class skews is warranted here. Particularly, given that the paper proposes to use this approaches in mobile which may work in both rapid and slow changing class skews. C] The Class Skew Detector is dependent on the base model. Thus, it is also likely that the empirical distribution estimated is biased and yet the final accuracies reported are much higher than the base model accuracies. There is something interesting happening here. An analysis of the robustness of the proposed approach in the face of noisy class skew detection could potentially make this paper a stronger work. D] The analysis in the paper has largely focused on pre-trained models. However, another analysis that could have been useful here is, varying the quality of the classifier (e.g. classifier trained on skewed training data vs. balanced training data) and measuring how the quality of the classifier correlates with the final performance. Maybe even attempt to answer the question "which classifiers are likely to work with this approach?" In fact, this analysis can be either done in a general context of any classifier or just CNN*s and identifying whether certain properties of CNN help in getting better performance. The paper lacks novelty and at the same time, it is not quite compensating that with a detailed analysis of the work. The problem is interesting and I like the work because the approach is simple and the results look good. I think with a stronger focus on more detailed analysis, this can be a good submission to an applied conference like MobiCom etc. By the way, the paper is riddled with several spelling errors - "filed" -> "field", page 1, second paragraph, last line "complimentary" -> "complementary", page 2, section 2, paragraph 1, last line "epoches" -> "epochs", page 2, section 2, transfer learning, second paragraph, second last line "CNNs does not use" -> "CNNs do not use", page 3, section 3, intuition, first paragraph, first line "formular" -> "formula", page 4, above equation 4 Equation 4 has a typo in the denominator, P_t(i) should be P_t(j), same with Equation 5 "obstained" -> "obtained", page 7, second paragraph, first line "adaptation" is almost everywhere spelled as "adaption"
assistant
 The paper proposes a simple idea to calibrate probabilities outputted by a CNN model to adapt easily to environments where class distributions change with space and time (and are often skewed). The paper shows that such a simple approach is sufficient to get good accuracies without requiring any costly retraining or transfer learning. Thereby proving to give benefits in terms of resource consumption and at the same time giving better results than the state of the art. However, A] The proposed calibration doesn*t take any CNN specific details into consideration, rather it is a general calibration method which was also proposed in Saerens et. al, 2002 (cited in the paper). It is unclear why the paper specifically talks about CNN. B] The proposed Class Skew Detector is a simple method. Change-point detection is a well-studied area. The paper lacks a literature review in this area and a reasoning of why the proposed approach is preferred. Also, an independent analysis of how the class skew detector behaves in the face of rapidly changing class skews versus slow changing class skews is warranted here. Particularly, given that the paper proposes to use this approaches in mobile which may work in both rapid and slow changing class skews. C] The Class Skew Detector is dependent on the base model. Thus, it is also likely that the empirical distribution estimated is biased and yet the final accuracies reported are much higher than the base model accuracies. There is something interesting happening here. An analysis of the robustness of the proposed approach in the face of noisy class skew detection could potentially make this paper a stronger work. D] The analysis in the paper has largely focused on pre-trained models. However, another analysis that could have been useful here is, varying the quality of the classifier (e.g. classifier trained on skewed training data vs. balanced training data) and measuring how the quality of the classifier correlates with the final performance. Maybe even attempt to answer the question "which classifiers are likely to work with this approach?" In fact, this analysis can be either done in a general context of any classifier or just CNN*s and identifying whether certain properties of CNN help in getting better performance. The paper lacks novelty and at the same time, it is not quite compensating that with a detailed analysis of the work. The problem is interesting and I like the work because the approach is simple and the results look good. I think with a stronger focus on more detailed analysis, this can be a good submission to an applied conference like MobiCom etc. By the way, the paper is riddled with several spelling errors - "filed" -> "field", page 1, second paragraph, last line "complimentary" -> "complementary", page 2, section 2, paragraph 1, last line "epoches" -> "epochs", page 2, section 2, transfer learning, second paragraph, second last line "CNNs does not use" -> "CNNs do not use", page 3, section 3, intuition, first paragraph, first line "formular" -> "formula", page 4, above equation 4 Equation 4 has a typo in the denominator, P_t(i) should be P_t(j), same with Equation 5 "obstained" -> "obtained", page 7, second paragraph, first line "adaptation" is almost everywhere spelled as "adaption"