Answers.Body: <p>I think what he means is that, while distribution <span class="math-container">$P(y  \mid x,s)$</span> has three variables, <span class="math-container">$P(y  \mid x)$</span> has two variables. The number of parameters required to describe a distribution (or samples to approximate it) grows exponentially with the number of variables (for more info see for instance Ian goodfellow, .. deep learning&quot;).</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2021-01-21T10:02:20.943Z
Answers.Text: I do not think this is correct.
Answers.UserDisplayName: The Pointer
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-12-24T03:24:23.987Z
Answers.LastActivityDate: 2021-01-06T09:50:30.573Z
Answers.LastEditDate: 2021-01-06T09:50:30.573Z
Answers.UserDisplayName: Reza_va
Body: <p>I am currently studying the paper <a href="https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.170&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer"><em>Learning and Evaluating Classifiers under Sample Selection Bias</em> by Bianca Zadrozny</a>. In section <strong>3.2. Logistic Regression</strong>, the author says the following:</p>
  <blockquote>
  <p><strong>3.2. Logistic regression</strong>
  In logistic regression, we use maximum likelihood to find the parameter vector <span class="math-container">$\beta$</span> of the following model:
  <span class="math-container">$$P(y = 1 \mid x) = \dfrac{1}{1 + \exp(\beta_0 + \beta_1 x_1 + \dots + \beta_n x_n)}$$</span>
  With sample selection bias we will instead fit:
  <span class="math-container">$$P(y = 1 \mid x, s = 1) = \dfrac{1}{1 + \exp(\beta_0 + \beta_1 x_1 + \dots + \beta_n x_n)}$$</span>
  However, because we are assuming that <span class="math-container">$y$</span> is independent of <span class="math-container">$s$</span> given <span class="math-container">$x$</span> we have that <span class="math-container">$P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$</span>. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced. Asymptotically, as long as <span class="math-container">$P(s = 1 \mid x)$</span> is greater than zero for all <span class="math-container">$x$</span>, the results on a selected sample approach the results on a random sample. In fact, this is true for any learner that models <span class="math-container">$P(y \mid x)$</span> directly. These are all local learners.</p>
  </blockquote>
  <p>This part is unclear to me:</p>
  <blockquote>
  <p>However, because we are assuming that <span class="math-container">$y$</span> is independent of <span class="math-container">$s$</span> given <span class="math-container">$x$</span> we have that <span class="math-container">$P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$</span>. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced.</p>
  </blockquote>
  <p>What is meant by &quot;the number of examples is reduced&quot;, and why is this the case?</p>
  
Tags: <classification><papers><logistic-regression><selection-bias>
Title: What is meant by "the number of examples is reduced", and why is this the case?
UserDisplayName: The Pointer
ThreadText: # What is meant by "the number of examples is reduced", and why is this the case?
  
  (Asked by: username_0 on 2020-12-19)
  
  I am currently studying the paper [*Learning and Evaluating Classifiers under Sample Selection Bias* by Bianca Zadrozny](https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.92.170&rep=rep1&type=pdf). In section **3.2. Logistic Regression**, the author says the following:
  
  > **3.2. Logistic regression**
  > In logistic regression, we use maximum likelihood to find the parameter vector $\beta$ of the following model:
  > $$P(y = 1 \mid x) = \dfrac{1}{1 + \exp(\beta\_0 + \beta\_1 x\_1 + \dots + \beta\_n x\_n)}$$
  > With sample selection bias we will instead fit:
  > $$P(y = 1 \mid x, s = 1) = \dfrac{1}{1 + \exp(\beta\_0 + \beta\_1 x\_1 + \dots + \beta\_n x\_n)}$$
  > However, because we are assuming that $y$ is independent of $s$ given $x$ we have that $P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced. Asymptotically, as long as $P(s = 1 \mid x)$ is greater than zero for all $x$, the results on a selected sample approach the results on a random sample. In fact, this is true for any learner that models $P(y \mid x)$ directly. These are all local learners.
  
  This part is unclear to me:
  
  > However, because we are assuming that $y$ is independent of $s$ given $x$ we have that $P(y = 1 \mid x, s = 1) = P(y = 1 \mid x)$. Thus, logistic regression is not affected by sample selection bias, except for the fact that the number of examples is reduced.
  
  What is meant by "the number of examples is reduced", and why is this the case?
  
  **Tags:** classification, papers, logistic-regression, selection-bias
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-12-24. Score: -1
  
  I think what he means is that, while distribution $P(y \mid x,s)$ has three variables, $P(y \mid x)$ has two variables. The number of parameters required to describe a distribution (or samples to approximate it) grows exponentially with the number of variables (for more info see for instance Ian goodfellow, .. deep learning").
  
  > **Comments:**
  > **username_0**: I do not think this is correct.  
  
  ---
  
  