Deep Hierarchical-Hyperspherical Learning (DH^2L)
Regularization is known to be an inexpensive and reasonable solution to alleviate over-fitting problems of inference models, including deep neural networks. In this paper, we propose a hierarchical regularization which preserves the semantic structure of a sample distribution. At the same time, this regularization promotes diversity by imposing distance between parameter vectors enlarged within semantic structures. To generate evenly distributed parameters, we constrain them to lie on emph{hierarchical hyperspheres}. Evenly distributed parameters are considered to be less redundant. To define hierarchical parameter space, we propose to reformulate the topology space with multiple hypersphere space. On each hypersphere space, the projection parameter is defined by two individual parameters. Since maximizing groupwise pairwise distance between points on hypersphere is nontrivial (generalized Thomson problem), we propose a new discrete metric integrated with continuous angle metric. Extensive experiments on publicly available datasets (CIFAR-10, CIFAR-100, CUB200-2011, and Stanford Cars), our proposed method shows improved generalization performance, especially when the number of super-classes is larger.
This paper proposes a regularization strategy motivated with principles of hierarchical, hyperspherical and discrete metric learning. Through regularization of as designed in level-wise, group-wise with the hierarchy of network, in their experiments with classification dataset, better performance are achieved with various distance. Pros: 1: I think the paper is well organized and motivated, the regularization of parameters in deep neural network is one of the center problem for effective learning. 2: The proposed strategy is effective with their experiments, various datasets and objective metrics are adopted to validate the regularization. Combination ablation study is sufficient. Cons: 1: The paper is also related with several popular normalization strategies such as weight normalization/standardization, group/batch normalization. It would be more convincing that some comparison could be performed against these strategies. 2: There would be better to show its performance using larger dataset such as ImageNet or COCO detection.
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Deep Hierarchical-Hyperspherical Learning (DH^2L)
Regularization is known to be an inexpensive and reasonable solution to alleviate over-fitting problems of inference models, including deep neural networks. In this paper, we propose a hierarchical regularization which preserves the semantic structure of a sample distribution. At the same time, this regularization promotes diversity by imposing distance between parameter vectors enlarged within semantic structures. To generate evenly distributed parameters, we constrain them to lie on emph{hierarchical hyperspheres}. Evenly distributed parameters are considered to be less redundant. To define hierarchical parameter space, we propose to reformulate the topology space with multiple hypersphere space. On each hypersphere space, the projection parameter is defined by two individual parameters. Since maximizing groupwise pairwise distance between points on hypersphere is nontrivial (generalized Thomson problem), we propose a new discrete metric integrated with continuous angle metric. Extensive experiments on publicly available datasets (CIFAR-10, CIFAR-100, CUB200-2011, and Stanford Cars), our proposed method shows improved generalization performance, especially when the number of super-classes is larger.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This paper proposes a regularization strategy motivated with principles of hierarchical, hyperspherical and discrete metric learning. Through regularization of as designed in level-wise, group-wise with the hierarchy of network, in their experiments with classification dataset, better performance are achieved with various distance. Pros: 1: I think the paper is well organized and motivated, the regularization of parameters in deep neural network is one of the center problem for effective learning. 2: The proposed strategy is effective with their experiments, various datasets and objective metrics are adopted to validate the regularization. Combination ablation study is sufficient. Cons: 1: The paper is also related with several popular normalization strategies such as weight normalization/standardization, group/batch normalization. It would be more convincing that some comparison could be performed against these strategies. 2: There would be better to show its performance using larger dataset such as ImageNet or COCO detection.
assistant
This paper proposes a regularization strategy motivated with principles of hierarchical, hyperspherical and discrete metric learning. Through regularization of as designed in level-wise, group-wise with the hierarchy of network, in their experiments with classification dataset, better performance are achieved with various distance. Pros: 1: I think the paper is well organized and motivated, the regularization of parameters in deep neural network is one of the center problem for effective learning. 2: The proposed strategy is effective with their experiments, various datasets and objective metrics are adopted to validate the regularization. Combination ablation study is sufficient. Cons: 1: The paper is also related with several popular normalization strategies such as weight normalization/standardization, group/batch normalization. It would be more convincing that some comparison could be performed against these strategies. 2: There would be better to show its performance using larger dataset such as ImageNet or COCO detection.