Answers.Body: <p>You are right that the baseline score is near zero only when there are a large number of label classes, i.e., when k is large. We should have qualified this line in the paper more carefully.</p>
  <p>In this sense, formally, the technique explains the *difference in prediction between the input score and the baseline score, as is made clear elsewhere in the paper (see Remark 1 and Proposition 1 for instance.)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-12T23:50:07.093Z
Answers.Text: I see that non 0 baseline does not affect the calculation of IG. Thanks for your clarification.
Answers.UserDisplayName: Taro Kiritani
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-07-10T18:48:29.950Z
Answers.LastActivityDate: 2020-07-10T18:48:29.950Z
Answers.UserDisplayName: Mukund Sundararajan
Body: <p>I am trying to understand <a href="https://arxiv.org/pdf/1703.01365.pdf" rel="nofollow noreferrer">Intagrated Gradients</a>, but have difficulty in understanding the authors' claim (in section 3, page 3):</p>
  <blockquote>
  <p>For most deep networks, it is possible to choose a baseline such that the prediction at the baseline is near zero (<span class="math-container">$F(x') \approx 0$</span>). (For image models, the black image baseline indeed satisfies this property.)</p>
  </blockquote>
  <p>They are talking about a function <span class="math-container">$F : R^n \rightarrow [0, 1]$</span> (in 2nd paragraph of section 3), and if you consider a deep learning classification model, the final layer would be a softmax layer. Then, I suspect for image models, the prediction at the baseline should be close to <span class="math-container">$1/k$</span>, where <span class="math-container">$k$</span> is the number of categories. For CIFAR10 and MNIST, this would equal to <span class="math-container">$1/10$</span>, which is not very close to <span class="math-container">$0$</span>. I have a binary classification model on which I am interested in applying the Integrated Gradients algorithm. Can the baseline output of <span class="math-container">$0.5$</span> be a problem?</p>
  <p>Another related question is, why did they choose a black image as the baseline in the first place? The parameters in image classification models (in a convolution layer) are typically initialized around <span class="math-container">$0$</span>, and the input is also normalized. Therefore, image classification models do not really care about the sign of inputs. I mean we could multiply all the training and test inputs with <span class="math-container">$-1$</span>, and the model would learn the task equivalently. I guess I can find other neutral images other than a black one. I suppose we could choose a white image as the baseline, or maybe the baseline should be all zero after normalization?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-21T13:46:02.047Z
Comments.Text: Regarding your last paragraph, in remark 1 (in the paper), they talk about the rationale behind a baseline and what it represents.
Comments.UserDisplayName: nbro
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2020-06-22T04:24:25.927Z
Comments.Text: @nbro I can see that a black image does imply "the absence of the cause", and it also might be "neutral". I still think a white image or random noise for example also could represent something neutral. In fact, I asked this question because the medical images I am working on have both positive and negative pixels, and the offset is kind of arbitrary. Maybe I will see mean pixel value and use it for the baseline.
Comments.UserDisplayName: Taro Kiritani
Tags: <deep-learning><image-recognition><papers>
Title: Why should the baseline's prediction be near zero, according to the Integrated Gradients paper?
UserDisplayName: Taro Kiritani
ThreadText: # Why should the baseline's prediction be near zero, according to the Integrated Gradients paper?
  
  (Asked by: username_0 on 2020-06-18)
  
  I am trying to understand [Intagrated Gradients](https://arxiv.org/pdf/1703.01365.pdf), but have difficulty in understanding the authors' claim (in section 3, page 3):
  
  > For most deep networks, it is possible to choose a baseline such that the prediction at the baseline is near zero ($F(x') \approx 0$). (For image models, the black image baseline indeed satisfies this property.)
  
  They are talking about a function $F : R^n \rightarrow [0, 1]$ (in 2nd paragraph of section 3), and if you consider a deep learning classification model, the final layer would be a softmax layer. Then, I suspect for image models, the prediction at the baseline should be close to $1/k$, where $k$ is the number of categories. For CIFAR10 and MNIST, this would equal to $1/10$, which is not very close to $0$. I have a binary classification model on which I am interested in applying the Integrated Gradients algorithm. Can the baseline output of $0.5$ be a problem?
  
  Another related question is, why did they choose a black image as the baseline in the first place? The parameters in image classification models (in a convolution layer) are typically initialized around $0$, and the input is also normalized. Therefore, image classification models do not really care about the sign of inputs. I mean we could multiply all the training and test inputs with $-1$, and the model would learn the task equivalently. I guess I can find other neutral images other than a black one. I suppose we could choose a white image as the baseline, or maybe the baseline should be all zero after normalization?
  
  **Tags:** deep-learning, image-recognition, papers
  > **Comments:**
  > **username_1**: Regarding your last paragraph, in remark 1 (in the paper), they talk about the rationale behind a baseline and what it represents.  
  > **username_0**: @nbro I can see that a black image does imply "the absence of the cause", and it also might be "neutral". I still think a white image or random noise for example also could represent something neutral. In fact, I asked this question because the medical images I am working on have both positive and negative pixels, and the offset is kind of arbitrary. Maybe I will see mean pixel value and use it for the baseline.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_2 on 2020-07-10. Score: 2 | Accepted answer
  
  You are right that the baseline score is near zero only when there are a large number of label classes, i.e., when k is large. We should have qualified this line in the paper more carefully.
  
  In this sense, formally, the technique explains the \*difference in prediction between the input score and the baseline score, as is made clear elsewhere in the paper (see Remark 1 and Proposition 1 for instance.)
  
  > **Comments:**
  > **username_0**: I see that non 0 baseline does not affect the calculation of IG. Thanks for your clarification.  
  
  ---
  
  