GroSS Decomposition: Group-Size Series Decomposition for Whole Search-Space Training
We present Group-size Series (GroSS) decomposition, a mathematical formulation of tensor factorisation into a series of approximations of increasing rank terms. GroSS allows for dynamic and differentiable selection of factorisation rank, which is analogous to a grouped convolution. Therefore, to the best of our knowledge, GroSS is the first method to simultaneously train differing numbers of groups within a single layer, as well as all possible combinations between layers. In doing so, GroSS trains an entire grouped convolution architecture search-space concurrently. We demonstrate this with a proof-of-concept exhaustive architecure search with a performance objective. GroSS represents a significant step towards liberating network architecture search from the burden of training and finetuning.
Summary: The authors introduce GROSS---a reformulation of block tensor decomposition, which allows multiple grouped convolutions (with varying group sizes) to be trained simultaneously. The basic idea is to reformulate the BTD so that higher-order decompositions can be expressed as functions of lower-order decompositions. Given this nesting, it is possible to implicitly train the lower-order decompositions while training the higher-order ones. The authors frame this contribution as a form of *neural architecture search* (NAS), arguing that this allows researchers to simultaneously train grouped-convolution CNNs with varying group sizes. After the simultaneous training based on the GROSS approach, the researcher can then select the group size that gives the best performance/accuracy tradeoff. The selected model can be further fine-tuned on the task, and the authors found that this improved performance. Empirical results on the CIFAR-10 dataset show that the proposed approach performs as expected, allowing for the simultaneous training of CNNs with grouped convolutions of varying orders. The results show the the proposed approach can find *better* solutions than a simple search over fixed architectures. Assessment: Overall, this is a well-written and soundly derived contribution. However, it is quite niche and---while the authors frame it as a form of NAS---in my view, this contribution is more in the realm of hyperparameter search for grouped convolutions, and not NAS in general. I would recommend reframing the introduction to make this fact more explicit, as the approach does not provide a general strategy for differentiable NAS. In addition, the empirical results are relatively shallow, with only one dataset and without detailed discussion of the variance of the results. Reasons to accept: - Well-written - Sound and well-motivated algorithm - Potential applications in cases where grouped convolutions are useful - Empirical results demonstrate validity of the proposed approach Reasons to reject: - Relatively niche contribution incorrectly framed as general contribution to NAS - Limited empirical analysis (e.g., only one dataset).
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
GroSS Decomposition: Group-Size Series Decomposition for Whole Search-Space Training
We present Group-size Series (GroSS) decomposition, a mathematical formulation of tensor factorisation into a series of approximations of increasing rank terms. GroSS allows for dynamic and differentiable selection of factorisation rank, which is analogous to a grouped convolution. Therefore, to the best of our knowledge, GroSS is the first method to simultaneously train differing numbers of groups within a single layer, as well as all possible combinations between layers. In doing so, GroSS trains an entire grouped convolution architecture search-space concurrently. We demonstrate this with a proof-of-concept exhaustive architecure search with a performance objective. GroSS represents a significant step towards liberating network architecture search from the burden of training and finetuning.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



Summary: The authors introduce GROSS---a reformulation of block tensor decomposition, which allows multiple grouped convolutions (with varying group sizes) to be trained simultaneously. The basic idea is to reformulate the BTD so that higher-order decompositions can be expressed as functions of lower-order decompositions. Given this nesting, it is possible to implicitly train the lower-order decompositions while training the higher-order ones. The authors frame this contribution as a form of *neural architecture search* (NAS), arguing that this allows researchers to simultaneously train grouped-convolution CNNs with varying group sizes. After the simultaneous training based on the GROSS approach, the researcher can then select the group size that gives the best performance/accuracy tradeoff. The selected model can be further fine-tuned on the task, and the authors found that this improved performance. Empirical results on the CIFAR-10 dataset show that the proposed approach performs as expected, allowing for the simultaneous training of CNNs with grouped convolutions of varying orders. The results show the the proposed approach can find *better* solutions than a simple search over fixed architectures. Assessment: Overall, this is a well-written and soundly derived contribution. However, it is quite niche and---while the authors frame it as a form of NAS---in my view, this contribution is more in the realm of hyperparameter search for grouped convolutions, and not NAS in general. I would recommend reframing the introduction to make this fact more explicit, as the approach does not provide a general strategy for differentiable NAS. In addition, the empirical results are relatively shallow, with only one dataset and without detailed discussion of the variance of the results. Reasons to accept: - Well-written - Sound and well-motivated algorithm - Potential applications in cases where grouped convolutions are useful - Empirical results demonstrate validity of the proposed approach Reasons to reject: - Relatively niche contribution incorrectly framed as general contribution to NAS - Limited empirical analysis (e.g., only one dataset).
assistant
Summary: The authors introduce GROSS---a reformulation of block tensor decomposition, which allows multiple grouped convolutions (with varying group sizes) to be trained simultaneously. The basic idea is to reformulate the BTD so that higher-order decompositions can be expressed as functions of lower-order decompositions. Given this nesting, it is possible to implicitly train the lower-order decompositions while training the higher-order ones. The authors frame this contribution as a form of *neural architecture search* (NAS), arguing that this allows researchers to simultaneously train grouped-convolution CNNs with varying group sizes. After the simultaneous training based on the GROSS approach, the researcher can then select the group size that gives the best performance/accuracy tradeoff. The selected model can be further fine-tuned on the task, and the authors found that this improved performance. Empirical results on the CIFAR-10 dataset show that the proposed approach performs as expected, allowing for the simultaneous training of CNNs with grouped convolutions of varying orders. The results show the the proposed approach can find *better* solutions than a simple search over fixed architectures. Assessment: Overall, this is a well-written and soundly derived contribution. However, it is quite niche and---while the authors frame it as a form of NAS---in my view, this contribution is more in the realm of hyperparameter search for grouped convolutions, and not NAS in general. I would recommend reframing the introduction to make this fact more explicit, as the approach does not provide a general strategy for differentiable NAS. In addition, the empirical results are relatively shallow, with only one dataset and without detailed discussion of the variance of the results. Reasons to accept: - Well-written - Sound and well-motivated algorithm - Potential applications in cases where grouped convolutions are useful - Empirical results demonstrate validity of the proposed approach Reasons to reject: - Relatively niche contribution incorrectly framed as general contribution to NAS - Limited empirical analysis (e.g., only one dataset).