Answers.Body: <p>I think, the proposed in the other answer CNN and RNN is a bad choice for this particular problem.</p>
  <p>The input is the unordered sequence of the features, corresponding to each runner, so the input is essentially a set structure without the notion of <strong>order</strong> and <strong>locality</strong>. If the runners are assigned a number in random order, there is no sense as runner 1 is close to 2, or runner 3 precedes runner 4.</p>
  <p>Therefore, <code>CNN</code> or <code>RNN</code> seem to be a bad choice since the add an inductive bias irrelevant to the data.</p>
  <p>The question is pretty old, and at that time <a href="http://papers.nips.cc/paper/7181-attention-is-all-you-%0Aneed.pdf" rel="nofollow noreferrer">Transformer architecture</a> was just invented, but it is the case, where it can be particularly well applied. Some points to note:</p>
  <ul>
  <li>Since there is no order, there is no need for positional embeddings</li>
  <li>Sequences are short, hence <span class="math-container">$O(N^2)$</span> complexity with respect to the length of sequence is small</li>
  <li>There is no need for decoder</li>
  </ul>
  <p>Overall, I would suggest to stack several encoder blocks and predict ranks from the resulting embeddings.</p>
  <p>As a loss, choose some reasonable choice from mentioned in the paper <a href="https://www.google.com/url?sa=t&amp;rct=j&amp;q=&amp;esrc=s&amp;source=web&amp;cd=&amp;ved=2ahUKEwiLprGMudb0AhUOuosKHfYiArYQFnoECAQQAQ&amp;url=https%3A%2F%2Fpapers.nips.cc%2Fpaper%2F3708-ranking-measures-and-loss-functions-in-learning-to-rank&amp;usg=AOvVaw2nUt88oYaPTMwc8oG-O22g" rel="nofollow noreferrer">Ranking Measures and Loss Functions in Learning to Rank</a>.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-12-09T09:53:45.633Z
Answers.LastActivityDate: 2021-12-09T09:53:45.633Z
Answers.UserDisplayName: spiridon_the_sun_rotator
Answers.Body: <p>The best option in your case would probably be zero-padding or padding up. This is simply zeroing out inputs for cases in which there is no data. It's done a lot on the borders of images for CNNs.</p>
  <p>Alternatively, you could just use an RNN, which can handle your variable-length inputs with ease.</p>
  
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-18T23:41:39.093Z
Answers.Text: Are there any examples or documentations? I’m using keras
Answers.UserDisplayName: Nickpick
Answers.ContentLicense: CC BY-SA 3.0
Answers.CreationDate: 2017-10-18T23:54:26.610Z
Answers.Text: Keras Docs on RNNs: https://keras.io/layers/recurrent/
  
  Github issue on this for Keras: https://github.com/fchollet/keras/issues/40
Answers.UserDisplayName: hisairnessag3
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2017-10-18T20:00:50.623Z
Answers.LastActivityDate: 2021-12-08T22:44:40.567Z
Answers.LastEditDate: 2021-12-08T22:44:40.567Z
Answers.UserDisplayName: hisairnessag3
Body: <p>I'm looking to design a neural network that can predict which runner wins in a sports game, where the number of runners varies between 2-10. In each case, specific data about the individual runners (for example, the weight, height, average speed in previous races, nationality, etc) would be fed into the neural network.</p>
  <p>What design would be most advantageous for such a neural network?</p>
  <p>Essentially this is a ranking problem where the number of inputs and outputs are variable.</p>
  
Tags: <neural-networks><recurrent-neural-networks><prediction><network-design>
Title: How to design a neural network when the number of inputs is variable?
UserDisplayName: Nickpick
ThreadText: # How to design a neural network when the number of inputs is variable?
  
  (Asked by: username_0 on 2017-10-16)
  
  I'm looking to design a neural network that can predict which runner wins in a sports game, where the number of runners varies between 2-10. In each case, specific data about the individual runners (for example, the weight, height, average speed in previous races, nationality, etc) would be fed into the neural network.
  
  What design would be most advantageous for such a neural network?
  
  Essentially this is a ranking problem where the number of inputs and outputs are variable.
  
  **Tags:** neural-networks, recurrent-neural-networks, prediction, network-design
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2017-10-18. Score: 4 | Accepted answer
  
  The best option in your case would probably be zero-padding or padding up. This is simply zeroing out inputs for cases in which there is no data. It's done a lot on the borders of images for CNNs.
  
  Alternatively, you could just use an RNN, which can handle your variable-length inputs with ease.
  
  > **Comments:**
  > **username_0**: Are there any examples or documentations? I’m using keras  
  > **username_2**: Keras Docs on RNNs: https://keras.io/layers/recurrent/  Github issue on this for Keras: https://github.com/fchollet/keras/issues/40  
  
  ---
  
  ### Answer by username_1 on 2021-12-09. Score: 1
  
  I think, the proposed in the other answer CNN and RNN is a bad choice for this particular problem.
  
  The input is the unordered sequence of the features, corresponding to each runner, so the input is essentially a set structure without the notion of **order** and **locality**. If the runners are assigned a number in random order, there is no sense as runner 1 is close to 2, or runner 3 precedes runner 4.
  
  Therefore, `CNN` or `RNN` seem to be a bad choice since the add an inductive bias irrelevant to the data.
  
  The question is pretty old, and at that time [Transformer architecture](http://papers.nips.cc/paper/7181-attention-is-all-you-%0Aneed.pdf) was just invented, but it is the case, where it can be particularly well applied. Some points to note:
  
  * Since there is no order, there is no need for positional embeddings
  * Sequences are short, hence $O(N^2)$ complexity with respect to the length of sequence is small
  * There is no need for decoder
  
  Overall, I would suggest to stack several encoder blocks and predict ranks from the resulting embeddings.
  
  As a loss, choose some reasonable choice from mentioned in the paper [Ranking Measures and Loss Functions in Learning to Rank](https://www.google.com/url?sa=t&rct=j&q=&esrc=s&source=web&cd=&ved=2ahUKEwiLprGMudb0AhUOuosKHfYiArYQFnoECAQQAQ&url=https%3A%2F%2Fpapers.nips.cc%2Fpaper%2F3708-ranking-measures-and-loss-functions-in-learning-to-rank&usg=AOvVaw2nUt88oYaPTMwc8oG-O22g).
  
  
  ---
  
  