Answers.Body: <p>check it</p>
  
  <pre><code>import keras
  from keras.layers import *
  
  X = np.array([[0,0], [0,1], [1,0], [1,1]])
  Y = np.array([[0], [0], [0], [1]])
  
  input = Input(shape=(2,))
  output = Dense(1, activation='sigmoid')(input)
  model = keras.Model(input, output)
  
  model.compile(keras.optimizers.Adam(1e0), 'binary_crossentropy', metrics=['acc'])
  model.fit(X, Y, epochs=10, batch_size=4, verbose=1)
  </code></pre>
  
  <p>which produces </p>
  
  <pre><code>Epoch 1/10
  4/4 [==============================] - 0s 52ms/step - loss: 0.7503 - acc: 0.7500
  Epoch 2/10
  4/4 [==============================] - 0s 817us/step - loss: 0.5142 - acc: 0.7500
  Epoch 3/10
  4/4 [==============================] - 0s 732us/step - loss: 0.4353 - acc: 0.7500
  Epoch 4/10
  4/4 [==============================] - 0s 694us/step - loss: 0.3413 - acc: 1.0000
  Epoch 5/10
  4/4 [==============================] - 0s 633us/step - loss: 0.2817 - acc: 1.0000
  Epoch 6/10
  4/4 [==============================] - 0s 679us/step - loss: 0.2299 - acc: 1.0000
  Epoch 7/10
  4/4 [==============================] - 0s 672us/step - loss: 0.1769 - acc: 1.0000
  Epoch 8/10
  4/4 [==============================] - 0s 721us/step - loss: 0.1412 - acc: 1.0000
  Epoch 9/10
  4/4 [==============================] - 0s 694us/step - loss: 0.1193 - acc: 1.0000
  Epoch 10/10
  4/4 [==============================] - 0s 716us/step - loss: 0.1015 - acc: 1.0000
  </code></pre>
  
  <p>...so yes, you can</p>
  
  <p>Also note you calculated marginal probabilities, here you want them conditioned on the input variables to actually solve for the parameters</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-04T13:51:59.960Z
Answers.Text: Thanks for the explanation.Had a doubt like if we have to check for the best fit of the model for logistic regression do we have to calculate it by using the Maximum Likelihood Estimate and getting the coefficients?
Answers.UserDisplayName: ten do
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-04T13:57:12.157Z
Answers.Text: so yeah you could think of it that way, given its form there is probably an analytical solution (unsure, just a guess), but for ease a simple optimizer will go a long way (especially because this is convex)
Answers.UserDisplayName: mshlis
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-04T11:39:03.000Z
Answers.LastActivityDate: 2019-06-04T11:39:03.000Z
Answers.UserDisplayName: mshlis
Body: <p>I have the following problem.</p>
  <blockquote>
  <p>A bank wants to decide whether a customer can be given a loan,
  based on two features related to (i) the monthly salary of the customer, and (ii) his/her account balance. For simplicity, we model the two features with two binary variables <span class="math-container">$X1$</span>, <span class="math-container">$X2$</span> and the class <span class="math-container">$Y$</span> (all of which can be either 0 or 1). <span class="math-container">$Y=1$</span> indicates that the customer can be given loan, and Y=0 indicates otherwise.</p>
  <p>Consider the following dataset having four instances:</p>
  <p>(<span class="math-container">$X1 = 0$</span>, <span class="math-container">$X2 = 0$</span>, <span class="math-container">$Y = 0$</span>)</p>
  <p>(<span class="math-container">$X1 = 0$</span>, <span class="math-container">$X2 = 1$</span>, <span class="math-container">$Y = 0$</span>)</p>
  <p>(<span class="math-container">$X1 = 1$</span>, <span class="math-container">$X2 = 0$</span>, <span class="math-container">$Y = 0$</span>)</p>
  <p>(<span class="math-container">$X1 = 1$</span>, <span class="math-container">$X2 = 1$</span>, <span class="math-container">$Y = 1$</span>)</p>
  <p>Can there be any logistic regression classifier using X1 and X2 as features, that can perfectly classify the given data?</p>
  </blockquote>
  <p>The approach followed in the question was to calculate respective probabilities for Y=0 and Y=1 respectively.  The value of <span class="math-container">$p$</span> obtained was <span class="math-container">$0.25$</span> and <span class="math-container">$(1-p)$</span> as <span class="math-container">$0.75$</span>. The <span class="math-container">$\log(p/1-p)$</span> is coming as negative.</p>
  <p>However, I don't understand what I need to do to understand whether there is a Logistic Regression classifier that can perfectly classify the given data.</p>
  
Tags: <machine-learning><logistic-regression>
Title: Is there a Logistic Regression classifier that can perfectly classify the given data in this problem?
UserDisplayName: ten do
ThreadText: # Is there a Logistic Regression classifier that can perfectly classify the given data in this problem?
  
  (Asked by: username_0 on 2019-06-04)
  
  I have the following problem.
  
  > A bank wants to decide whether a customer can be given a loan,
  > based on two features related to (i) the monthly salary of the customer, and (ii) his/her account balance. For simplicity, we model the two features with two binary variables $X1$, $X2$ and the class $Y$ (all of which can be either 0 or 1). $Y=1$ indicates that the customer can be given loan, and Y=0 indicates otherwise.
  >
  > Consider the following dataset having four instances:
  >
  > ($X1 = 0$, $X2 = 0$, $Y = 0$)
  >
  > ($X1 = 0$, $X2 = 1$, $Y = 0$)
  >
  > ($X1 = 1$, $X2 = 0$, $Y = 0$)
  >
  > ($X1 = 1$, $X2 = 1$, $Y = 1$)
  >
  > Can there be any logistic regression classifier using X1 and X2 as features, that can perfectly classify the given data?
  
  The approach followed in the question was to calculate respective probabilities for Y=0 and Y=1 respectively. The value of $p$ obtained was $0.25$ and $(1-p)$ as $0.75$. The $\log(p/1-p)$ is coming as negative.
  
  However, I don't understand what I need to do to understand whether there is a Logistic Regression classifier that can perfectly classify the given data.
  
  **Tags:** machine-learning, logistic-regression
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-04. Score: 1 | Accepted answer
  
  check it
  
  ```
  import keras
  from keras.layers import *
  
  X = np.array([[0,0], [0,1], [1,0], [1,1]])
  Y = np.array([[0], [0], [0], [1]])
  
  input = Input(shape=(2,))
  output = Dense(1, activation='sigmoid')(input)
  model = keras.Model(input, output)
  
  model.compile(keras.optimizers.Adam(1e0), 'binary_crossentropy', metrics=['acc'])
  model.fit(X, Y, epochs=10, batch_size=4, verbose=1)
  
  ```
  
  which produces
  
  ```
  Epoch 1/10
  4/4 [==============================] - 0s 52ms/step - loss: 0.7503 - acc: 0.7500
  Epoch 2/10
  4/4 [==============================] - 0s 817us/step - loss: 0.5142 - acc: 0.7500
  Epoch 3/10
  4/4 [==============================] - 0s 732us/step - loss: 0.4353 - acc: 0.7500
  Epoch 4/10
  4/4 [==============================] - 0s 694us/step - loss: 0.3413 - acc: 1.0000
  Epoch 5/10
  4/4 [==============================] - 0s 633us/step - loss: 0.2817 - acc: 1.0000
  Epoch 6/10
  4/4 [==============================] - 0s 679us/step - loss: 0.2299 - acc: 1.0000
  Epoch 7/10
  4/4 [==============================] - 0s 672us/step - loss: 0.1769 - acc: 1.0000
  Epoch 8/10
  4/4 [==============================] - 0s 721us/step - loss: 0.1412 - acc: 1.0000
  Epoch 9/10
  4/4 [==============================] - 0s 694us/step - loss: 0.1193 - acc: 1.0000
  Epoch 10/10
  4/4 [==============================] - 0s 716us/step - loss: 0.1015 - acc: 1.0000
  
  ```
  
  ...so yes, you can
  
  Also note you calculated marginal probabilities, here you want them conditioned on the input variables to actually solve for the parameters
  
  > **Comments:**
  > **username_0**: Thanks for the explanation.Had a doubt like if we have to check for the best fit of the model for logistic regression do we have to calculate it by using the Maximum Likelihood Estimate and getting the coefficients?  
  > **username_1**: so yeah you could think of it that way, given its form there is probably an analytical solution (unsure, just a guess), but for ease a simple optimizer will go a long way (especially because this is convex)  
  
  ---
  
  