Answers.Body: <p>If you only care about whether an image contains a a certain digit after arbitrary rotation, not where the digit is located, rotation invariance suffices and the latest <a href="https://arxiv.org/abs/2211.11812" rel="noreferrer">Rotation-Invariant Coordinate-CNNs</a> as recommended in the comment could be a simple straightforward approach without rotated data augmentations during training in theory, simply based on the input space's polar coordinate system, where after filter convolutions in polar space aggregating over the angular axis makes the representation invariant to the specific angle of rotation.</p>
  <p>However, in deep learning <a href="https://arxiv.org/abs/2205.07362" rel="noreferrer">equivariance</a> is a more important and useful property than invariance as Cohen &amp; Welling emphasized in their 2016 paper on the <a href="https://arxiv.org/pdf/1602.07576" rel="noreferrer">G-CNNs</a> to address similar rotation issue albeit in a group-theoretic way.</p>
  <blockquote>
  <p>the network or layer <span class="math-container">$Φ$</span> that maps one representation to another should be structure preserving. For G-spaces this means that <span class="math-container">$Φ$</span> has to be equivariant: <span class="math-container">$Φ(T_g x)=T′_gΦ(x)$</span> (1). That is, transforming an input <span class="math-container">$x$</span> by a transformation <span class="math-container">$g$</span> (forming <span class="math-container">$T_g x$</span>)... Equivariance can be realized in many ways, and in particular the operators <span class="math-container">$T$</span> and <span class="math-container">$T′$</span> need not be the same... From equation (1) we see that the familiar concept of invariance is a <strong>special</strong> kind of equivariance where <span class="math-container">$T′_g$</span> is the identity transformation for all <span class="math-container">$g$</span>. In deep learning, general equivariance is more useful than invariance because it is impossible to determine if features are in the right spatial configuration if they are invariant.</p>
  </blockquote>
  <p>Therefore in many real world CNN deep learning facial-recognition and object-detection tasks rotation-equivariance is required instead of mere invariance for classification tasks. For example, for self-driving cars knowing the relative location of the pedestrian is more important than knowing a pedestrian exists. If you rotate <em>and</em> shift the captured image, the feature maps in a CNN should also rotate and shift correspondingly preserving the structure of features. In such cases Cohen &amp; Welling's 2016 <a href="https://arxiv.org/pdf/1612.08498" rel="noreferrer">steerable CNNs</a> could be kept in mind for <em>many</em> types of equivariance preservation including rotation, translation and reflection. Here's another <a href="https://maurice-weiler.gitlab.io/blog_post/cnn-book_3_steerable_cnns" rel="noreferrer">blog</a> dedicated to such topic.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-12-24T22:03:34.577Z
Answers.LastActivityDate: 2024-12-24T22:03:34.577Z
Answers.UserDisplayName: cinch
Body: <p>Training an MNIST classifier with a regular ANN will make the model recognize its unrotated version.</p>
  <p>But is there such a model where I train the unrotated version as usual, but it also recognizes its rotated version, e.g., the 90-degree version, a.k.a. the transposed version that I never fed the rotated version during training before?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-12-24T15:14:18.317Z
Comments.Text: this https://arxiv.org/abs/2211.11812
Comments.UserDisplayName: Alberto
Tags: <image-recognition><models>
Title: Which model would recognize the rotated version of its input without explicit training during inference?
UserDisplayName: Muhammad Ikhwan Perwira
ThreadText: # Which model would recognize the rotated version of its input without explicit training during inference?
  
  (Asked by: username_0 on 2024-12-24)
  
  Training an MNIST classifier with a regular ANN will make the model recognize its unrotated version.
  
  But is there such a model where I train the unrotated version as usual, but it also recognizes its rotated version, e.g., the 90-degree version, a.k.a. the transposed version that I never fed the rotated version during training before?
  
  **Tags:** image-recognition, models
  > **Comments:**
  > **username_1**: this https://arxiv.org/abs/2211.11812  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2024-12-24. Score: 5 | Accepted answer
  
  If you only care about whether an image contains a a certain digit after arbitrary rotation, not where the digit is located, rotation invariance suffices and the latest [Rotation-Invariant Coordinate-CNNs](https://arxiv.org/abs/2211.11812) as recommended in the comment could be a simple straightforward approach without rotated data augmentations during training in theory, simply based on the input space's polar coordinate system, where after filter convolutions in polar space aggregating over the angular axis makes the representation invariant to the specific angle of rotation.
  
  However, in deep learning [equivariance](https://arxiv.org/abs/2205.07362) is a more important and useful property than invariance as Cohen & Welling emphasized in their 2016 paper on the [G-CNNs](https://arxiv.org/pdf/1602.07576) to address similar rotation issue albeit in a group-theoretic way.
  
  > the network or layer $Φ$ that maps one representation to another should be structure preserving. For G-spaces this means that $Φ$ has to be equivariant: $Φ(T\_g x)=T′\_gΦ(x)$ (1). That is, transforming an input $x$ by a transformation $g$ (forming $T\_g x$)... Equivariance can be realized in many ways, and in particular the operators $T$ and $T′$ need not be the same... From equation (1) we see that the familiar concept of invariance is a **special** kind of equivariance where $T′\_g$ is the identity transformation for all $g$. In deep learning, general equivariance is more useful than invariance because it is impossible to determine if features are in the right spatial configuration if they are invariant.
  
  Therefore in many real world CNN deep learning facial-recognition and object-detection tasks rotation-equivariance is required instead of mere invariance for classification tasks. For example, for self-driving cars knowing the relative location of the pedestrian is more important than knowing a pedestrian exists. If you rotate *and* shift the captured image, the feature maps in a CNN should also rotate and shift correspondingly preserving the structure of features. In such cases Cohen & Welling's 2016 [steerable CNNs](https://arxiv.org/pdf/1612.08498) could be kept in mind for *many* types of equivariance preservation including rotation, translation and reflection. Here's another [blog](https://maurice-weiler.gitlab.io/blog_post/cnn-book_3_steerable_cnns) dedicated to such topic.
  
  
  ---
  
  