Sequenced-Replacement Sampling for Deep Learning
We propose sequenced-replacement sampling (SRS) for training deep neural networks. The basic idea is to assign a fixed sequence index to each sample in the dataset. Once a mini-batch is randomly drawn in each training iteration, we refill the original dataset by successively adding samples according to their sequence index. Thus we carry out replacement sampling but in a batched and sequenced way. In a sense, SRS could be viewed as a way of performing "mini-batch augmentation". It is particularly useful for a task where we have a relatively small images-per-class such as CIFAR-100. Together with a longer period of initial large learning rate, it significantly improves the classification accuracy in CIFAR-100 over the current state-of-the-art results. Our experiments indicate that training deeper networks with SRS is less prone to over-fitting. In the best case, we achieve an error rate as low as 10.10%.
 The paper suggests a new way of sampling mini-batches for training deep neural nets. The idea is to first index the samples then select the batches during training in a sequential way. The proposed method is tested on the CIFAR dataset and some improvement on the classification accuracy is reported. I find the idea interesting but feel that much more is needed in order to have a better understanding of how the proposed method works, when it works and when it doesn*t work. Some theoretical insight, or at least a more systematic experimental study, is needed to justify the proposed method.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Sequenced-Replacement Sampling for Deep Learning
We propose sequenced-replacement sampling (SRS) for training deep neural networks. The basic idea is to assign a fixed sequence index to each sample in the dataset. Once a mini-batch is randomly drawn in each training iteration, we refill the original dataset by successively adding samples according to their sequence index. Thus we carry out replacement sampling but in a batched and sequenced way. In a sense, SRS could be viewed as a way of performing "mini-batch augmentation". It is particularly useful for a task where we have a relatively small images-per-class such as CIFAR-100. Together with a longer period of initial large learning rate, it significantly improves the classification accuracy in CIFAR-100 over the current state-of-the-art results. Our experiments indicate that training deeper networks with SRS is less prone to over-fitting. In the best case, we achieve an error rate as low as 10.10%.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



 The paper suggests a new way of sampling mini-batches for training deep neural nets. The idea is to first index the samples then select the batches during training in a sequential way. The proposed method is tested on the CIFAR dataset and some improvement on the classification accuracy is reported. I find the idea interesting but feel that much more is needed in order to have a better understanding of how the proposed method works, when it works and when it doesn*t work. Some theoretical insight, or at least a more systematic experimental study, is needed to justify the proposed method.
assistant
 The paper suggests a new way of sampling mini-batches for training deep neural nets. The idea is to first index the samples then select the batches during training in a sequential way. The proposed method is tested on the CIFAR dataset and some improvement on the classification accuracy is reported. I find the idea interesting but feel that much more is needed in order to have a better understanding of how the proposed method works, when it works and when it doesn*t work. Some theoretical insight, or at least a more systematic experimental study, is needed to justify the proposed method.