Answers.Body: <ol>
  <li><p>Localization network is not trained separately on special transform of input. It's just a part of feed forward network which is trained as a whole, with normal backpropagation.</p></li>
  <li><p>It's just part of the network which affect the loss function. As in any backpropagation loss function propagate back gradient, which backpropagate through final part of the network, backpropagate through differentiable sampler(that is non-trivial part, which use transfromation produced by localization subnetwork) and after that backpropagate into localization part</p></li>
  <li><p>Whole approach of spatial transformers could be in doubt now. There were some anecdotal evidences that it was not working on less trivial tasks (private communications, was not working for me either)</p></li>
  </ol>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-26T07:03:10.460Z
Answers.Text: Thanks for the reply. However my doubt still stands. How does the regression layer output the exact transform parameters in a previously unseen image?  It seems magic to me. Am I missing something about regression which the localization network uses?
Answers.UserDisplayName: Abhisek Dash
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-26T07:07:19.757Z
Answers.Text: It's called "regression layer" because it produce not discrete set of probabilities but continuous parameters. It's just common fully connected layer. Nothing special about it. And it seems whole method work only on transformation of simple flat images like road signs. It's not so good for affine approximation, not a magical solution
Answers.UserDisplayName: mirror2image
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-02-26T06:58:08.810Z
Answers.LastActivityDate: 2020-02-26T07:03:34.230Z
Answers.LastEditDate: 2020-02-26T07:03:34.230Z
Answers.UserDisplayName: mirror2image
Body: <p>I am trying to understand the <strong>spatial transformer network</strong> mentioned in this paper <a href="https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf" rel="nofollow noreferrer">https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf</a>. <strong>I am clear about the last two stages of the spatial transformer i.e. the grid generator and sampler</strong>. However I am <strong>unable to understand the localization network</strong> which outputs the parameters of the transformation that is applied to the input image. So here are my doubts.</p>
  
  <ol>
  <li>Is the network trained on various affine/projective transforms of the input or only the standard input with a standard pose?</li>
  <li>If the answer to question 1 is no, then how does the regression layer correctly regress the values of the transformation applied to the image? In other words how does the regression layer know what transformation parameters are required when it has never seen those inputs before?</li>
  </ol>
  
  <p>Thanks in advance. </p>
  
Tags: <neural-networks><machine-learning><regression><transformer>
Title: How does the regression layer in the localization network of a spatial transformer work?
UserDisplayName: Abhisek Dash
ThreadText: # How does the regression layer in the localization network of a spatial transformer work?
  
  (Asked by: username_0 on 2020-02-26)
  
  I am trying to understand the **spatial transformer network** mentioned in this paper <https://papers.nips.cc/paper/5854-spatial-transformer-networks.pdf>. **I am clear about the last two stages of the spatial transformer i.e. the grid generator and sampler**. However I am **unable to understand the localization network** which outputs the parameters of the transformation that is applied to the input image. So here are my doubts.
  
  1. Is the network trained on various affine/projective transforms of the input or only the standard input with a standard pose?
  2. If the answer to question 1 is no, then how does the regression layer correctly regress the values of the transformation applied to the image? In other words how does the regression layer know what transformation parameters are required when it has never seen those inputs before?
  
  Thanks in advance.
  
  **Tags:** neural-networks, machine-learning, regression, transformer
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-02-26. Score: 0
  
  1. Localization network is not trained separately on special transform of input. It's just a part of feed forward network which is trained as a whole, with normal backpropagation.
  2. It's just part of the network which affect the loss function. As in any backpropagation loss function propagate back gradient, which backpropagate through final part of the network, backpropagate through differentiable sampler(that is non-trivial part, which use transfromation produced by localization subnetwork) and after that backpropagate into localization part
  3. Whole approach of spatial transformers could be in doubt now. There were some anecdotal evidences that it was not working on less trivial tasks (private communications, was not working for me either)
  
  > **Comments:**
  > **username_0**: Thanks for the reply. However my doubt still stands. How does the regression layer output the exact transform parameters in a previously unseen image?  It seems magic to me. Am I missing something about regression which the localization network uses?  
  > **username_1**: It's called "regression layer" because it produce not discrete set of probabilities but continuous parameters. It's just common fully connected layer. Nothing special about it. And it seems whole method work only on transformation of simple flat images like road signs. It's not so good for affine approximation, not a magical solution  
  
  ---
  
  