Answers.Body: <p>Note: This is always going to be an estimate until you actually run the experiment. ML is not always predictable.</p>
  <p>If order truly does not matter, then I think it will be better to design a network architecture that automatically ignores order, instead of using one that cares about order and then training it to ignore order. If nothing else, less training data will be needed since you don't need to train it on permutations of the input - similar to why CNNs are useful for image recognition.</p>
  <p>One network architecture could be to have an &quot;input processing block&quot; (a group of layers) and an &quot;output processing block&quot;. First apply the input processing block to each input. Then add together all the outputs of the input processing blocks. Because addition is insensitive to order, this step completely discards any information about the order. Finally, apply the output processing block to the sum of those and the output from that block is your final output.</p>
  <pre><code>           output
               ^
               |
             +--+
             |NN| (different NN to the one below)
             +--+
               ^
               |
  +---------------------------+
  | add all together          |
  +---------------------------+
   ^    ^    ^    ^    ^    ^
   |    |    |    |    |    |
  +--+ +--+ +--+ +--+ +--+ +--+
  |NN| |NN| |NN| |NN| |NN| |NN| (same NN 6 times)
  +--+ +--+ +--+ +--+ +--+ +--+
   ^    ^    ^    ^    ^    ^
   |    |    |    |    |    |
   C    E    D    B    F    A
  </code></pre>
  <p>This is just one idea. You are not limited to addition; you can use any commutative operation in the middle. Even something like an attention layer could be used.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-09-20T12:56:21.843Z
Answers.LastActivityDate: 2021-09-20T12:56:21.843Z
Answers.UserDisplayName: Criticize SE actions means ban
Body: <p>I am a beginner with DL. I did some tutorials and I know the basics of TensorFlow. But I have a problem understanding how to construct more advanced NNs.</p>
  <p>Let's say I have 6 inputs and a list of 500 names from which you can pick any, but only 6 at the time. The output should be one value between <span class="math-container">$0.0$</span> and <span class="math-container">$1.0$</span>.</p>
  <p>My question is, how I can handle random order in inputs? In inputs 1-6 you have names <strong>ABCDEF</strong> and the output score is 0.7. I need the same output if input will be in order <strong>CEDBFA</strong>. How can I handle this? Should I make random shuffle on inputs during training, or should I make for every output value 500D binary vector-like <span class="math-container">$[0,0,1,0,1,...,0,1,0,0,0]$</span>, where index position in the array is the corresponding token of name and then feed it in 500 inputs? Or there is some better way?</p>
  
Tags: <neural-networks><tensorflow><keras><input-layer>
Title: How to handle random order of inputs and get same output?
UserDisplayName: tech2097
ThreadText: # How to handle random order of inputs and get same output?
  
  (Asked by: username_0 on 2021-09-19)
  
  I am a beginner with DL. I did some tutorials and I know the basics of TensorFlow. But I have a problem understanding how to construct more advanced NNs.
  
  Let's say I have 6 inputs and a list of 500 names from which you can pick any, but only 6 at the time. The output should be one value between $0.0$ and $1.0$.
  
  My question is, how I can handle random order in inputs? In inputs 1-6 you have names **ABCDEF** and the output score is 0.7. I need the same output if input will be in order **CEDBFA**. How can I handle this? Should I make random shuffle on inputs during training, or should I make for every output value 500D binary vector-like $[0,0,1,0,1,...,0,1,0,0,0]$, where index position in the array is the corresponding token of name and then feed it in 500 inputs? Or there is some better way?
  
  **Tags:** neural-networks, tensorflow, keras, input-layer
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2021-09-20. Score: 1
  
  Note: This is always going to be an estimate until you actually run the experiment. ML is not always predictable.
  If order truly does not matter, then I think it will be better to design a network architecture that automatically ignores order, instead of using one that cares about order and then training it to ignore order. If nothing else, less training data will be needed since you don't need to train it on permutations of the input - similar to why CNNs are useful for image recognition.
  One network architecture could be to have an "input processing block" (a group of layers) and an "output processing block". First apply the input processing block to each input. Then add together all the outputs of the input processing blocks. Because addition is insensitive to order, this step completely discards any information about the order. Finally, apply the output processing block to the sum of those and the output from that block is your final output.
             output
               ^
               |
             +--+
             |NN| (different NN to the one below)
             +--+
               ^
               |
  +---------------------------+
  | add all together          |
  +---------------------------+
   ^    ^    ^    ^    ^    ^
   |    |    |    |    |    |
  +--+ +--+ +--+ +--+ +--+ +--+
  |NN| |NN| |NN| |NN| |NN| |NN| (same NN 6 times)
  +--+ +--+ +--+ +--+ +--+ +--+
   ^    ^    ^    ^    ^    ^
   |    |    |    |    |    |
   C    E    D    B    F    A
  
  This is just one idea. You are not limited to addition; you can use any commutative operation in the middle. Even something like an attention layer could be used.
  
  
  ---
  
  