Answers.Body: <p>To test hypothesis 1 the only way is to try different architectures and/or approaches. Basically brutal grid search.</p>
  <p>To test hypothesis 2 the way to go is data exploration and feature importance analysis. You can compute different correlation scores (Pearson, Spearman, Chi-square) between each individual feature and your target, plus correlation between each pair of features. Comparing to the target gives you an idea of which features are strong predictors and which are weak ones, comparing features tells you how much redundancy is present within you training data.</p>
  <p>Keep in mind that decision tree based models allow you to automatically assess which features are strong predictors, so training a random forest or xgboost is one good way to do both things in one shot: training a model and assess your data quality.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-15T11:44:48.330Z
Answers.LastActivityDate: 2022-08-15T11:44:48.330Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>given a ML model that performs poorly, can you differentiate between those two causes:</p>
  <ol>
  <li>bad architecture/not enough data</li>
  <li>the features do not hold enough information to solve the problem</li>
  </ol>
  <p>I'm interested in identifying num 2., what analysis can I perform to prove that num 2. is occurring when solving some ML problem?
  Context is supervised learning (and the model is not necessarily a neural net)</p>
  
Tags: <machine-learning><supervised-learning>
Title: Identify features don't hold required information in a ML problem
UserDisplayName: Gilad Deutsch
ThreadText: # Identify features don't hold required information in a ML problem
  
  (Asked by: username_0 on 2022-08-11)
  
  given a ML model that performs poorly, can you differentiate between those two causes:
  
  1. bad architecture/not enough data
  2. the features do not hold enough information to solve the problem
  
  I'm interested in identifying num 2., what analysis can I perform to prove that num 2. is occurring when solving some ML problem?
  Context is supervised learning (and the model is not necessarily a neural net)
  
  **Tags:** machine-learning, supervised-learning
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-08-15. Score: 2
  
  To test hypothesis 1 the only way is to try different architectures and/or approaches. Basically brutal grid search.
  
  To test hypothesis 2 the way to go is data exploration and feature importance analysis. You can compute different correlation scores (Pearson, Spearman, Chi-square) between each individual feature and your target, plus correlation between each pair of features. Comparing to the target gives you an idea of which features are strong predictors and which are weak ones, comparing features tells you how much redundancy is present within you training data.
  
  Keep in mind that decision tree based models allow you to automatically assess which features are strong predictors, so training a random forest or xgboost is one good way to do both things in one shot: training a model and assess your data quality.
  
  
  ---
  
  