Answers.Body: <blockquote>
  <p>Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.</p>
  </blockquote>
  <p>This will help inform you how much redundancy there is between A and B. However, even if you can predict B with 100% accuracy from A, you <em>may</em> still be better off using A+B (instead of A alone) to predict C.</p>
  <blockquote>
  <p>If I get good predictions when testing the model A -&gt; B, is it true then that adding B to A in predicting C is completely useless ?</p>
  </blockquote>
  <p>It is an indicator that adding B <em>probably</em> won't make great improvements to your prediction of C.</p>
  <p>The only way to be sure is to make the model that uses A+B and compare its performance against a model that uses only A. If collecting B costs time or other resources, then perform this check by limiting both models to only learn from the subset of data where you have all of A, B, C available.</p>
  <blockquote>
  <p>And furthermore, are there smarter ways to decide if/when a feature is useless ?</p>
  </blockquote>
  <p>Another thing you can do is to try and predict C from B alone. It doesn't need to score well, but may indicate that something useful is in the data if you get better than chance results. Scoring badly unfortunately doesn't rule it out for working well in combination with A.</p>
  <p>Generally, if you cannot reason it clearly one way or another from theory, the accepted method is to build variations of your model with and without a feature and test them. You do have to be aware of the <a href="https://www.tylervigen.com/spurious-correlations" rel="nofollow noreferrer">chances of spurious correlation</a> though, so usually you need to have some basis or motivation for considering any new feature, from your domain knowledge that applies to the model.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-10T08:59:44.310Z
Answers.LastActivityDate: 2020-07-10T09:21:49.937Z
Answers.LastEditDate: 2020-07-10T09:21:49.937Z
Answers.UserDisplayName: Neil Slater
Body: <p>I'm building a model, where, from a feature set A, I want to predict a target set C. I need to understand if another feature set B, together with A, can improve my model performances, instead of using only A.</p>
  <p>Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.</p>
  <p>If I get good predictions when testing the model A -&gt; B, is it true then that adding B to A in predicting C is completely useless?</p>
  <p>And furthermore, are there smarter ways to decide if/when a feature is useless?</p>
  
Tags: <neural-networks><deep-learning><prediction><feature-selection><features>
Title: When is adding a feature useless?
UserDisplayName: Cla
ThreadText: # When is adding a feature useless?
  
  (Asked by: username_0 on 2020-07-10)
  
  I'm building a model, where, from a feature set A, I want to predict a target set C. I need to understand if another feature set B, together with A, can improve my model performances, instead of using only A.
  
  Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.
  
  If I get good predictions when testing the model A -> B, is it true then that adding B to A in predicting C is completely useless?
  
  And furthermore, are there smarter ways to decide if/when a feature is useless?
  
  **Tags:** neural-networks, deep-learning, prediction, feature-selection, features
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-07-10. Score: 2 | Accepted answer
  
  > Now I want to check if I can predict B directly from A, since, in my understanding, this would mean that info on B is already inside A.
  
  This will help inform you how much redundancy there is between A and B. However, even if you can predict B with 100% accuracy from A, you *may* still be better off using A+B (instead of A alone) to predict C.
  
  > If I get good predictions when testing the model A -> B, is it true then that adding B to A in predicting C is completely useless ?
  
  It is an indicator that adding B *probably* won't make great improvements to your prediction of C.
  
  The only way to be sure is to make the model that uses A+B and compare its performance against a model that uses only A. If collecting B costs time or other resources, then perform this check by limiting both models to only learn from the subset of data where you have all of A, B, C available.
  
  > And furthermore, are there smarter ways to decide if/when a feature is useless ?
  
  Another thing you can do is to try and predict C from B alone. It doesn't need to score well, but may indicate that something useful is in the data if you get better than chance results. Scoring badly unfortunately doesn't rule it out for working well in combination with A.
  
  Generally, if you cannot reason it clearly one way or another from theory, the accepted method is to build variations of your model with and without a feature and test them. You do have to be aware of the [chances of spurious correlation](https://www.tylervigen.com/spurious-correlations) though, so usually you need to have some basis or motivation for considering any new feature, from your domain knowledge that applies to the model.
  
  
  ---
  
  