Answers.Body: <p>There are lots of domains in which <em>set-valued</em> (<em>permutation-invariant</em>) output is needed: for example, inferring bounding boxes in image object detection, or audio source separation when the output should be multiple separated waveforms.</p>
  <p>From the latter, a clear example is this 2016 <a href="https://arxiv.org/abs/1607.00325" rel="nofollow noreferrer">&quot;permutation invariant training&quot;</a>, whose method is quite simple: during training you calculate the loss corresponding to <em>each</em> permutation, then use the lowest value as the loss for backpropagation. This is a method that is still used. It should be much better than your averaging-based approach, because the latter would give a lot of additive components corresponding to the &quot;bad&quot; permutations, which are highly-varying but also irrelevant to the correct solution.</p>
  <p>In this method, the system doesn't learn how to <em>infer</em> the correct permutation.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-11-10T22:00:05.227Z
Answers.LastActivityDate: 2024-11-10T22:00:05.227Z
Answers.UserDisplayName: Dan Stowell
Body: <p>I am trying to train a neural network that predicts 2 3D points in space by looking at the hit counts from voxel data. I am currently doing this by making the network return 8 floats (x1,y1,z1,x2,y2,z2,dist1,dist2) where x1,y1,z1 and x2,y2,z2 are unit vectors of the point and dist1 and dist2 are their distances to the origin.</p>
  <p>Actual ordering of the output does not matter. So I tried taking average of the possible permutations as loss. However constraining the network to first return the closest point to the origin made the model better.</p>
  <p>But now when the points that I want to predict have similar distance to the origin (like 5 and 5.2) model gets confused and makes errors.</p>
  <p>So how can I resolve this issue? Is there a better way to order the data?</p>
  
Tags: <neural-networks><machine-learning><convolutional-neural-networks><long-short-term-memory>
Title: Teaching a neural network when order of output does not matter
UserDisplayName: efemantarci
ThreadText: # Teaching a neural network when order of output does not matter
  
  (Asked by: username_0 on 2024-11-10)
  
  I am trying to train a neural network that predicts 2 3D points in space by looking at the hit counts from voxel data. I am currently doing this by making the network return 8 floats (x1,y1,z1,x2,y2,z2,dist1,dist2) where x1,y1,z1 and x2,y2,z2 are unit vectors of the point and dist1 and dist2 are their distances to the origin.
  
  Actual ordering of the output does not matter. So I tried taking average of the possible permutations as loss. However constraining the network to first return the closest point to the origin made the model better.
  
  But now when the points that I want to predict have similar distance to the origin (like 5 and 5.2) model gets confused and makes errors.
  
  So how can I resolve this issue? Is there a better way to order the data?
  
  **Tags:** neural-networks, machine-learning, convolutional-neural-networks, long-short-term-memory
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-11-10. Score: 1 | Accepted answer
  
  There are lots of domains in which *set-valued* (*permutation-invariant*) output is needed: for example, inferring bounding boxes in image object detection, or audio source separation when the output should be multiple separated waveforms.
  
  From the latter, a clear example is this 2016 ["permutation invariant training"](https://arxiv.org/abs/1607.00325), whose method is quite simple: during training you calculate the loss corresponding to *each* permutation, then use the lowest value as the loss for backpropagation. This is a method that is still used. It should be much better than your averaging-based approach, because the latter would give a lot of additive components corresponding to the "bad" permutations, which are highly-varying but also irrelevant to the correct solution.
  
  In this method, the system doesn't learn how to *infer* the correct permutation.
  
  
  ---
  
  