Answers.Body: <p>If you know it is symmetric, then you could do a couple things.</p>
  <ol>
  <li>Zero out a half.</li>
  </ol>
  <p>Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output matrix and just have the network regress the other half. Just don't make the network do more work than it needs to do.</p>
  <ol start="2">
  <li>Learn both, but add symmetric loss</li>
  </ol>
  <p>In your case, it looks like you could create two loss functions added together.</p>
  <p><span class="math-container">$focal(x, y) + focal(x^T, y)$</span></p>
  <p>This will help the network learn both halves equally.</p>
  <ol start="3">
  <li>L1 between <span class="math-container">$x$</span> and <span class="math-container">$x^T$</span></li>
  </ol>
  <p>This might be silly but adding a Huber loss between <span class="math-container">$x$</span> and <span class="math-container">$x^T$</span> might help promote symmetry, but I'm not as much if a fan of it. Personally I'm more partial to (1).</p>
  <ol start="4">
  <li>Combine (1) and (2)</li>
  </ol>
  <p>You could take the loss function of (2) at train time but at test time just use whatever half had better metrics and copy that to the bottom half.</p>
  <ol start="5">
  <li>Post Processing</li>
  </ol>
  <p>Add a post processing custom layer that takes the average of the two halves so that you guarantee symmetry.</p>
  <p><span class="math-container">$x' = \frac{1}{2}(x + x^T)$</span></p>
  <p>Then do your normal focal loss. Personally I like this the best since it always guarantees a symmetric output and is a pretty easy custom layer.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-30T16:48:04.877Z
Answers.LastActivityDate: 2021-02-04T18:28:02.937Z
Answers.LastEditDate: 2021-02-04T18:28:02.937Z
Answers.UserDisplayName: juicedatom
Body: <p>I have a task of extremely sparse binary segmentation, i.e. the segmentation mask contains either 0 or 1, and there are ~95% zeros and only ~5% ones. I use the <a href="https://arxiv.org/abs/1708.02002" rel="nofollow noreferrer">focal loss</a> to address the sparseness (which is equivalent in my case to imbalances). I have another piece of information that I want to incorporate in the loss term.</p>
  <p>The desired output is always symmetric over the diagonal. I was searching for a way to use this information in the loss, but I couldn't find a solution. How would I do this?</p>
  <p>For some example of the symmetry in the segmentation maps, I added an arrow to show the axis of symmetry:</p>
  <p><a href="https://i.sstatic.net/SuS85.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/SuS85.png" alt="enter image description here" /></a></p>
  <p><a href="https://i.sstatic.net/i48Av.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/i48Av.png" alt="enter image description here" /></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-01-26T13:21:08.510Z
Comments.Text: You could add a term that sums (or averages depending on your exact preference) the difference between both triangle parts of the matrix (self-regulating loss, as it does not need a true prediction).
Comments.UserDisplayName: Christian Steinmeyer
Tags: <convolutional-neural-networks><objective-functions><pytorch><image-segmentation>
Title: How to incorporate a symmetry constraint in the loss function to train a CNN?
UserDisplayName: David
ThreadText: # How to incorporate a symmetry constraint in the loss function to train a CNN?
  
  (Asked by: username_0 on 2021-01-26)
  
  I have a task of extremely sparse binary segmentation, i.e. the segmentation mask contains either 0 or 1, and there are ~95% zeros and only ~5% ones. I use the [focal loss](https://arxiv.org/abs/1708.02002) to address the sparseness (which is equivalent in my case to imbalances). I have another piece of information that I want to incorporate in the loss term.
  
  The desired output is always symmetric over the diagonal. I was searching for a way to use this information in the loss, but I couldn't find a solution. How would I do this?
  
  For some example of the symmetry in the segmentation maps, I added an arrow to show the axis of symmetry:
  
  [![enter image description here](https://i.sstatic.net/SuS85.png)](https://i.sstatic.net/SuS85.png)
  
  [![enter image description here](https://i.sstatic.net/i48Av.png)](https://i.sstatic.net/i48Av.png)
  
  **Tags:** convolutional-neural-networks, objective-functions, pytorch, image-segmentation
  
  > **Comments:**
  > **username_1**: You could add a term that sums (or averages depending on your exact preference) the difference between both triangle parts of the matrix (self-regulating loss, as it does not need a true prediction).  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2021-01-30. Score: 2 | Accepted answer
  
  If you know it is symmetric, then you could do a couple things.
  
  1. Zero out a half.
  
  Don't bother learning both halves of the image. Just put a zero mask over the upper or lower half of the output matrix and just have the network regress the other half. Just don't make the network do more work than it needs to do.
  
  2. Learn both, but add symmetric loss
  
  In your case, it looks like you could create two loss functions added together.
  
  $focal(x, y) + focal(x^T, y)$
  
  This will help the network learn both halves equally.
  
  3. L1 between $x$ and $x^T$
  
  This might be silly but adding a Huber loss between $x$ and $x^T$ might help promote symmetry, but I'm not as much if a fan of it. Personally I'm more partial to (1).
  
  4. Combine (1) and (2)
  
  You could take the loss function of (2) at train time but at test time just use whatever half had better metrics and copy that to the bottom half.
  
  5. Post Processing
  
  Add a post processing custom layer that takes the average of the two halves so that you guarantee symmetry.
  
  $x' = \frac{1}{2}(x + x^T)$
  
  Then do your normal focal loss. Personally I like this the best since it always guarantees a symmetric output and is a pretty easy custom layer.
  
  
  ---
  
  