Body: <p>Suppose I have a neural network with 5 inputs: <code>[A,B,C,D,E]</code></p>
  <p>There is only 1 output. The expected accuracy of the model should increase when all 5 inputs are available, but often not all 5 inputs are available. For example, I might have case where I only have a variable number of the inputs, e.g. <code>[A,B,C,-,-]</code>, <code>[A,-,-,-,E]</code>, <code>[-,B,-,D,-]</code>, <code>[A,-,-,-,-]</code>, <code>[-,-,C,-,-]</code>, <code>[A,-,C,D,E]</code>, etc.</p>
  <p>In such a situation, what is the best way to train or build the neural network? Are there any specific approaches or architectures recommended for this type of problem?</p>
  <p>A couple ideas that come to mind include:</p>
  <ol>
  <li><p>Double the number of inputs to the neural network by including a second &quot;binary input vector&quot; that determines whether the input variable is present or not. For example, the binary input vector for the inputs <code>[A,-,C,-,E]</code> would simply correspond to <code>[1,0,1,0,1]</code>, which could be fed into the neural network as well. The outstanding question is how does one treat the undefined variables with &quot;-&quot; as placeholders in such an example...perhaps defaulting to 0 for &quot;-&quot; is one naive but simple way when coupling the binary vector.</p>
  </li>
  <li><p>Build and train separate neural network for every combination of <code>[A,B,C,D,E]</code> — this could certainly be implemented, but would be a brute force approach that requires a lot of training and be rather inefficient. For 5 variables, this would require <a href="https://math.stackexchange.com/a/161568">31 separate neural networks</a>, and would scale poorly as the number of potential input variables increase</p>
  </li>
  </ol>
  <p>Any and all insights into this problem are appreciated!</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-15T13:00:24.617Z
Comments.Text: Some domain knowledge would help here, what is the nature of these inputs? For example are they real numbers, with Gaussian distribution? Or are they always non-negative?
Comments.UserDisplayName: NikoNyrh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-15T13:46:22.440Z
Comments.Text: Neural network or not the typical way of dealing with missing variables is imputation (mean, mode, etc).  Increasing the dimensionality of the problem (by adding variables) is tempting the curse of dimensionality.
Comments.UserDisplayName: foreverska
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-15T16:45:42.963Z
Comments.Text: Oh, and is each input a single number, or a tensor? And do they have any co-correlation or are they basically independent from each other?
Comments.UserDisplayName: NikoNyrh
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-16T00:22:42.363Z
Comments.Text: @NikoNyrh each variable input (e.g. A, B, C, D, E) corresponds to a real number between -1 and +1. The exact distribution of each variable is not precisely known and likely not Gaussian (@foreverska thus perhaps it's best to not assume anything about its distribution)
Comments.UserDisplayName: user18959
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-19T03:23:15.807Z
Comments.Text: @foreverska is there any further context that would be helpful? For added reference, these inputs (i.e. A, B, C, D, E) correspond to specific types of measurements. For example, A = height, B = age, etc. and thus I don't believe randomly ordering inputs would be a successful approach
Comments.UserDisplayName: user18959
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-03-19T12:47:43.773Z
Comments.Text: The fact that one or more variables are missing regularly makes it a sparse dataset. Depending on how sparse, imputation can help (via mean, etc).  One may also try a method to make it dense (PCA, etc).  Then maybe exotic NN strategies, they’re just function approximators though, data science still applies.
Comments.UserDisplayName: foreverska
Tags: <neural-networks><machine-learning><deep-learning><features><input-layer>
Title: What is the best way to train a neural network with a variable number of inputs?
UserDisplayName: user18959
ThreadText: # What is the best way to train a neural network with a variable number of inputs?
  
  (Asked by: username_0 on 2024-03-15)
  
  Suppose I have a neural network with 5 inputs: `[A,B,C,D,E]`
  
  There is only 1 output. The expected accuracy of the model should increase when all 5 inputs are available, but often not all 5 inputs are available. For example, I might have case where I only have a variable number of the inputs, e.g. `[A,B,C,-,-]`, `[A,-,-,-,E]`, `[-,B,-,D,-]`, `[A,-,-,-,-]`, `[-,-,C,-,-]`, `[A,-,C,D,E]`, etc.
  
  In such a situation, what is the best way to train or build the neural network? Are there any specific approaches or architectures recommended for this type of problem?
  
  A couple ideas that come to mind include:
  
  1. Double the number of inputs to the neural network by including a second "binary input vector" that determines whether the input variable is present or not. For example, the binary input vector for the inputs `[A,-,C,-,E]` would simply correspond to `[1,0,1,0,1]`, which could be fed into the neural network as well. The outstanding question is how does one treat the undefined variables with "-" as placeholders in such an example...perhaps defaulting to 0 for "-" is one naive but simple way when coupling the binary vector.
  2. Build and train separate neural network for every combination of `[A,B,C,D,E]` — this could certainly be implemented, but would be a brute force approach that requires a lot of training and be rather inefficient. For 5 variables, this would require [31 separate neural networks](https://math.stackexchange.com/a/161568), and would scale poorly as the number of potential input variables increase
  
  Any and all insights into this problem are appreciated!
  
  **Tags:** neural-networks, machine-learning, deep-learning, features, input-layer
  
  > **Comments:**
  > **username_1**: Some domain knowledge would help here, what is the nature of these inputs? For example are they real numbers, with Gaussian distribution? Or are they always non-negative?  
  > **username_2**: Neural network or not the typical way of dealing with missing variables is imputation (mean, mode, etc).  Increasing the dimensionality of the problem (by adding variables) is tempting the curse of dimensionality.  
  > **username_1**: Oh, and is each input a single number, or a tensor? And do they have any co-correlation or are they basically independent from each other?  
  > **username_0**: @username_1 each variable input (e.g. A, B, C, D, E) corresponds to a real number between -1 and +1. The exact distribution of each variable is not precisely known and likely not Gaussian (@username_2 thus perhaps it's best to not assume anything about its distribution)  
  > **username_0**: @username_2 is there any further context that would be helpful? For added reference, these inputs (i.e. A, B, C, D, E) correspond to specific types of measurements. For example, A = height, B = age, etc. and thus I don't believe randomly ordering inputs would be a successful approach  
  > **username_2**: The fact that one or more variables are missing regularly makes it a sparse dataset. Depending on how sparse, imputation can help (via mean, etc).  One may also try a method to make it dense (PCA, etc).  Then maybe exotic NN strategies, they’re just function approximators though, data science still applies.  
  
  ---
  
  ## 0 answers
  
  