Dynamic Steerable Frame Networks
Filters in a convolutional network are typically parametrized in a pixel basis. As an orthonormal basis, pixels may represent any arbitrary vector in Rn. In this paper, we relax this orthonormality requirement and extend the set of viable bases to the generalized notion of frames. When applying suitable frame bases to ResNets on Cifar-10+ we demonstrate improved error rates by substitution only. By exploiting the transformation properties of such generalized bases, we arrive at steerable frames, that allow to continuously transform CNN filters under arbitrary Lie-groups. Further allowing us to locally separate pose from canonical appearance. We implement this in the Dynamic Steerable Frame Network, that dynamically estimates the transformations of filters, conditioned on its input. The derived method presents a hybrid of Dynamic Filter Networks and Spatial Transformer Networks that can be implemented in any convolutional architecture, as we illustrate in two examples. First, we illustrate estimation properties of steerable frames with a Dynamic Steerable Frame Network, compared to a Dynamic Filter Network on the task of edge detection, where we show clear advantages of the derived steerable frames. Lastly, we insert the Dynamic Steerable Frame Network as a module in a convolutional LSTM on the task of limited-data hand-gesture recognition from video and illustrate effective dynamic regularization and show clear advantages over Spatial Transformer Networks. In this paper, we have laid out the foundations of Frame-based convolutional networks and Dynamic Steerable Frame Networks while illustrating their advantages for continuously transforming features and data-efficient learning.
This works applies steerable frames for various tasks where convolutional neural networks with location invariant operators are traditionally applied. Authors provide a detailed overview of steerable frames followed with an experimental section which applies dynamic steerable network to small machine learning problems where the steerability is conceptually useful. Even though the evaluation is performed only on few small tasks, the reason why more tasks were not evaluated is that piece-wise pose invariance is needed only for a subset of tasks. The fact, that simply using overcomplete bases as a sort of *feature pre-processing* improves the results for already highly optimized ResNet and DenseNet architectures is quite interesting achievement. For the edge detection, a relatively hard baseline is selected - the Dynamic Filter Networks, which already attempts to achieve position invariant filters. The fact that DSFN improves the performance on this task verifies that regressing the parametrization of the steerable filters yields better results than regressing the filters directly. In the last experiment authors apply the network to video classification using LSTMs and they show that the improved performance is not due to increased capacity of the network. In general, it is quite interesting work. Even though it does not offer ground-breaking results (mainly in a sense of not performing experiments on larger tasks), it is theoretically interesting and shows promising results. There are few minor issues and suggestions related to the paper: * For the LSTM experiment, in order to be more exact, it would be useful to include information about total number of parameters, as the network which estimates the pose also increases the number of parameters. * Would it be possible to provide more details about how the back-propagation is done through the steerable filters? * For the Edge Detection experiment, it would be useful to provide results for some standard baseline - e.g. CNN with a similar number of parameters. Simply to see how useful it is to have location-variant filters for this task. * The last sentence in second paragraph on page 1 is missing a verb. Also it is maybe unnecessary. * The hyphenation for ConvNet is incorrect on multiple places (probably `hyphenation{Conv-Net}` would fix it).
You are a member of the scientific community tasked with peer review. Review the following paper title and abstract.
### Title + Abstract
Dynamic Steerable Frame Networks
Filters in a convolutional network are typically parametrized in a pixel basis. As an orthonormal basis, pixels may represent any arbitrary vector in Rn. In this paper, we relax this orthonormality requirement and extend the set of viable bases to the generalized notion of frames. When applying suitable frame bases to ResNets on Cifar-10+ we demonstrate improved error rates by substitution only. By exploiting the transformation properties of such generalized bases, we arrive at steerable frames, that allow to continuously transform CNN filters under arbitrary Lie-groups. Further allowing us to locally separate pose from canonical appearance. We implement this in the Dynamic Steerable Frame Network, that dynamically estimates the transformations of filters, conditioned on its input. The derived method presents a hybrid of Dynamic Filter Networks and Spatial Transformer Networks that can be implemented in any convolutional architecture, as we illustrate in two examples. First, we illustrate estimation properties of steerable frames with a Dynamic Steerable Frame Network, compared to a Dynamic Filter Network on the task of edge detection, where we show clear advantages of the derived steerable frames. Lastly, we insert the Dynamic Steerable Frame Network as a module in a convolutional LSTM on the task of limited-data hand-gesture recognition from video and illustrate effective dynamic regularization and show clear advantages over Spatial Transformer Networks. In this paper, we have laid out the foundations of Frame-based convolutional networks and Dynamic Steerable Frame Networks while illustrating their advantages for continuously transforming features and data-efficient learning.
user
<think>Okay, to do a decent peer review, I should start with summarizing what the work is about. This includes novelty, contributions, clarity, and impact. I might include suggestions to issues that I find critical to the usefulness of the work. Starting with novelty, there are different types of novelty. The first aspect of novelty is scope and scale. Is the work covering a broader perspective than previous works? The second aspect is methodology. Is the proposed methodology novel compared to previous methods? Do they provide any significant advantage over previous methodologies? The third aspect is results. Are the results different compared previous work? Does this lead to any novel theories or an extension of current theories? Next is clarity, I will be reading the work in depth and analyze if any sentences can be potentially ambiguous or misleading. If any sentences are ambiguous, I will state them and give a suggestion on how to rephrase such sentence or how to make more clear to the audience. Lastly, impact is important. I will attempt to predict how such work will impact the scientific community. This might include the applications of such work and how the work might transform the way we study a certain topic. Finally, I will might only mention issues that are critical. This is to keep the review as concise and helpful as possible.</think>



This works applies steerable frames for various tasks where convolutional neural networks with location invariant operators are traditionally applied. Authors provide a detailed overview of steerable frames followed with an experimental section which applies dynamic steerable network to small machine learning problems where the steerability is conceptually useful. Even though the evaluation is performed only on few small tasks, the reason why more tasks were not evaluated is that piece-wise pose invariance is needed only for a subset of tasks. The fact, that simply using overcomplete bases as a sort of *feature pre-processing* improves the results for already highly optimized ResNet and DenseNet architectures is quite interesting achievement. For the edge detection, a relatively hard baseline is selected - the Dynamic Filter Networks, which already attempts to achieve position invariant filters. The fact that DSFN improves the performance on this task verifies that regressing the parametrization of the steerable filters yields better results than regressing the filters directly. In the last experiment authors apply the network to video classification using LSTMs and they show that the improved performance is not due to increased capacity of the network. In general, it is quite interesting work. Even though it does not offer ground-breaking results (mainly in a sense of not performing experiments on larger tasks), it is theoretically interesting and shows promising results. There are few minor issues and suggestions related to the paper: * For the LSTM experiment, in order to be more exact, it would be useful to include information about total number of parameters, as the network which estimates the pose also increases the number of parameters. * Would it be possible to provide more details about how the back-propagation is done through the steerable filters? * For the Edge Detection experiment, it would be useful to provide results for some standard baseline - e.g. CNN with a similar number of parameters. Simply to see how useful it is to have location-variant filters for this task. * The last sentence in second paragraph on page 1 is missing a verb. Also it is maybe unnecessary. * The hyphenation for ConvNet is incorrect on multiple places (probably `hyphenation{Conv-Net}` would fix it).
assistant
This works applies steerable frames for various tasks where convolutional neural networks with location invariant operators are traditionally applied. Authors provide a detailed overview of steerable frames followed with an experimental section which applies dynamic steerable network to small machine learning problems where the steerability is conceptually useful. Even though the evaluation is performed only on few small tasks, the reason why more tasks were not evaluated is that piece-wise pose invariance is needed only for a subset of tasks. The fact, that simply using overcomplete bases as a sort of *feature pre-processing* improves the results for already highly optimized ResNet and DenseNet architectures is quite interesting achievement. For the edge detection, a relatively hard baseline is selected - the Dynamic Filter Networks, which already attempts to achieve position invariant filters. The fact that DSFN improves the performance on this task verifies that regressing the parametrization of the steerable filters yields better results than regressing the filters directly. In the last experiment authors apply the network to video classification using LSTMs and they show that the improved performance is not due to increased capacity of the network. In general, it is quite interesting work. Even though it does not offer ground-breaking results (mainly in a sense of not performing experiments on larger tasks), it is theoretically interesting and shows promising results. There are few minor issues and suggestions related to the paper: * For the LSTM experiment, in order to be more exact, it would be useful to include information about total number of parameters, as the network which estimates the pose also increases the number of parameters. * Would it be possible to provide more details about how the back-propagation is done through the steerable filters? * For the Edge Detection experiment, it would be useful to provide results for some standard baseline - e.g. CNN with a similar number of parameters. Simply to see how useful it is to have location-variant filters for this task. * The last sentence in second paragraph on page 1 is missing a verb. Also it is maybe unnecessary. * The hyphenation for ConvNet is incorrect on multiple places (probably `hyphenation{Conv-Net}` would fix it).