Answers.Body: <p>Two popular methods I’ve seen done:  </p>
  
  <p><strong>1)</strong> For each feature, remove it and run the model and see the impact it has on the result. The idea is that the larger the impact, the more pertinent it was to the result.   </p>
  
  <p><strong>2)</strong> Look at the gradients magnitude <span class="math-container">$|\nabla_f {y} |$</span>. You can either look at the raw gradient or look at the <a href="https://www.cs.toronto.edu/~guerzhoy/321/lec/W07/HowConvNetsSee.pdf" rel="nofollow noreferrer">guided back-propagation</a> which is just the back props  product rule, but you only look at when the nodes positively help trigger a neuron by taking only the positive gradients at each step.</p>
  
  <p>There’s probably also more methods. Hope this helped.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-29T17:29:00.140Z
Answers.Text: We don't get all of the (nonlinear) interactions in vanilla linear regression, but #1 is how we do parameter inference in statistics! (At least it's a way.)
Answers.UserDisplayName: Dave
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-07-09T03:17:44.193Z
Answers.LastActivityDate: 2019-07-09T03:17:44.193Z
Answers.UserDisplayName: mshlis
Body: <p>I have a simple neural network for binary classification.</p>
  <p>The input features include age, sex, economic situation, illness, disability, etc. The output is simply 1 and 0.</p>
  <p>I would like to order the features from the greatest to least impact it had on the classification.</p>
  <p>An example answer could look like this:</p>
  <p>Classification: 1</p>
  <ol>
  <li>illness</li>
  <li>economic situation</li>
  <li>disability</li>
  <li>sex</li>
  <li>age</li>
  </ol>
  <p>Another example:</p>
  <p>Classification: 0</p>
  <ol>
  <li>economic situation</li>
  <li>age</li>
  <li>disability</li>
  <li>sex</li>
  <li>illness</li>
  </ol>
  
Tags: <neural-networks><classification><binary-classification><features><inference>
Title: When doing binary classification with neural networks, how can I order the importance of the features for a class?
UserDisplayName: Ricardo Prieto
ThreadText: # When doing binary classification with neural networks, how can I order the importance of the features for a class?
  
  (Asked by: username_0 on 2019-07-09)
  
  I have a simple neural network for binary classification.
  
  The input features include age, sex, economic situation, illness, disability, etc. The output is simply 1 and 0.
  
  I would like to order the features from the greatest to least impact it had on the classification.
  
  An example answer could look like this:
  
  Classification: 1
  
  1. illness
  2. economic situation
  3. disability
  4. sex
  5. age
  
  Another example:
  
  Classification: 0
  
  1. economic situation
  2. age
  3. disability
  4. sex
  5. illness
  
  **Tags:** neural-networks, classification, binary-classification, features, inference
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-07-09. Score: 1
  
  Two popular methods I’ve seen done:
  
  **1)** For each feature, remove it and run the model and see the impact it has on the result. The idea is that the larger the impact, the more pertinent it was to the result.
  
  **2)** Look at the gradients magnitude $|\nabla\_f {y} |$. You can either look at the raw gradient or look at the [guided back-propagation](https://www.cs.toronto.edu/~guerzhoy/321/lec/W07/HowConvNetsSee.pdf) which is just the back props product rule, but you only look at when the nodes positively help trigger a neuron by taking only the positive gradients at each step.
  
  There’s probably also more methods. Hope this helped.
  
  > **Comments:**
  > **username_2**: We don't get all of the (nonlinear) interactions in vanilla linear regression, but #1 is how we do parameter inference in statistics! (At least it's a way.)  
  
  ---
  
  