Answers.Body: <p>Based on the clarifications given in the comments on the original question, i will try my best to give an answer.</p>
  <p><strong>If the number of features increases, does the approximation capability of an ANN improve theoretically speaking?</strong>
  It depends on whether this additional feature adds 'usefulness' to the data that was already supported. If you add a feature that is already in the dataset, it obviously will not increase the approximation capabilities. If you add a feature that is not yet in the current set of features and does influence the thing you are trying to approximate, then yes! There are some exceptions in this case of course, such as what if features are multicorrelated etc. It is not super important or measurable in ANNs, but statistics has a bunch of theory on this stuff. Simple statistical multiple regression has 4 assumptions and also checks for multicollinearity to see whether it is possible get 'useful' results from the data etc. As ANNs is super general, this is not applicable, but you could argue that similar practices could be looked into when seeking the best possible 'theoretical' validity of the features that you are using.</p>
  <p><strong>Does adding more features make your ANNs produce more accurate results (in practice)?</strong>
  Again, it depends. If you indeed added a useful feature, then the possibility exists that your ANN will in practice be able to produce more accurate results. But if your ANN is not able to converge to a solution, then your results will be gibberish. It is more likely that a network is unable to converge if you just throw data at it. More data requires bigger networks etc. So it is highly dependent on your training method. AKA, if your added feature is useful and the method you use is sufficient, then I'd argue that, yes, adding a feature will most likely make your ANNs produce more accurate results.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-05T15:48:58.617Z
Answers.Text: Are you sure that the approximation capabilities **of neural networks** change depending on the number of features? According to which "theory" is this true? For example, does the VC dimension of a neural network change if you change the number or type of features?
Answers.UserDisplayName: nbro
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-06T09:48:30.040Z
Answers.Text: I am not very knowledged about the VC dimensions theory or similar theories that apply more generally to NNs. If you are, please do elaborate on them. My approach here was to give a semi-intuitive understanding of when to expect 'more' approximation capabilities. I guess you could argue that my first stance on the 'theoretical' side could also be put as more 'practical', but I don't think it matters too much here.
Answers.UserDisplayName: Robin van Hoorn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-05T13:04:16.347Z
Answers.LastActivityDate: 2020-12-05T13:04:16.347Z
Answers.UserDisplayName: Robin van Hoorn
Body: <p>What is the impact of the number of features on the prediction power of an ANN model (in general)? Does an increase in the number of features mean a more powerful prediction model (for approximation purpose)?</p>
  <p>I'm asking these questions because I am wondering if there is any benefit in using two variables (rather than one) to predict one output.</p>
  <p>If there is a scientific paper that answers my question, I would thank you.</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-04T14:14:53.753Z
Comments.Text: I'm not sure 1. if you're looking for a _theoretical_ result on the _approximation capabilities_ of neural networks when the number of features changes, or 2. if you're just asking if adding features can, in practice, make your NN produce more accurate results (e.g. in terms of accuracy). Can you clarify this?
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-12-04T18:19:53.827Z
Comments.Text: yes, I mean 1 and 2
Comments.UserDisplayName: Yazan Alatoom
Tags: <neural-networks><deep-learning><reference-request><feature-selection><features>
Title: What is the impact of the number of features on the prediction power of a neural network?
UserDisplayName: Yazan Alatoom
ThreadText: # What is the impact of the number of features on the prediction power of a neural network?
  
  (Asked by: username_0 on 2020-12-03)
  
  What is the impact of the number of features on the prediction power of an ANN model (in general)? Does an increase in the number of features mean a more powerful prediction model (for approximation purpose)?
  
  I'm asking these questions because I am wondering if there is any benefit in using two variables (rather than one) to predict one output.
  
  If there is a scientific paper that answers my question, I would thank you.
  
  **Tags:** neural-networks, deep-learning, reference-request, feature-selection, features
  > **Comments:**
  > **username_1**: I'm not sure 1. if you're looking for a _theoretical_ result on the _approximation capabilities_ of neural networks when the number of features changes, or 2. if you're just asking if adding features can, in practice, make your NN produce more accurate results (e.g. in terms of accuracy). Can you clarify this?  
  > **username_0**: yes, I mean 1 and 2  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-12-05. Score: 0 | Accepted answer
  
  Based on the clarifications given in the comments on the original question, i will try my best to give an answer.
  
  **If the number of features increases, does the approximation capability of an ANN improve theoretically speaking?**
  It depends on whether this additional feature adds 'usefulness' to the data that was already supported. If you add a feature that is already in the dataset, it obviously will not increase the approximation capabilities. If you add a feature that is not yet in the current set of features and does influence the thing you are trying to approximate, then yes! There are some exceptions in this case of course, such as what if features are multicorrelated etc. It is not super important or measurable in ANNs, but statistics has a bunch of theory on this stuff. Simple statistical multiple regression has 4 assumptions and also checks for multicollinearity to see whether it is possible get 'useful' results from the data etc. As ANNs is super general, this is not applicable, but you could argue that similar practices could be looked into when seeking the best possible 'theoretical' validity of the features that you are using.
  
  **Does adding more features make your ANNs produce more accurate results (in practice)?**
  Again, it depends. If you indeed added a useful feature, then the possibility exists that your ANN will in practice be able to produce more accurate results. But if your ANN is not able to converge to a solution, then your results will be gibberish. It is more likely that a network is unable to converge if you just throw data at it. More data requires bigger networks etc. So it is highly dependent on your training method. AKA, if your added feature is useful and the method you use is sufficient, then I'd argue that, yes, adding a feature will most likely make your ANNs produce more accurate results.
  
  > **Comments:**
  > **username_1**: Are you sure that the approximation capabilities **of neural networks** change depending on the number of features? According to which "theory" is this true? For example, does the VC dimension of a neural network change if you change the number or type of features?  
  > **username_2**: I am not very knowledged about the VC dimensions theory or similar theories that apply more generally to NNs. If you are, please do elaborate on them. My approach here was to give a semi-intuitive understanding of when to expect 'more' approximation capabilities. I guess you could argue that my first stance on the 'theoretical' side could also be put as more 'practical', but I don't think it matters too much here.  
  
  ---
  
  