Body: <p>Apologies for the lengthy title.
  My question is about the weight update rule for logistic regression using stochastic gradient descent.</p>
  
  <p>I have just started experimenting on Logistic Regression. I came across two weight update expressions and did not know which one is more accurate and why they are different.</p>
  
  <p><strong>The first Method:</strong></p>
  
  <p>Source: (Book) Artificial Intelligence: A Modern Approach by Norvig, Russell
   on page 726-727:
  using the L2 loss function:</p>
  
  <p><a href="https://i.sstatic.net/QA8O9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QA8O9.png" alt="enter image description here"></a></p>
  
  <p><a href="https://i.sstatic.net/FwVLC.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FwVLC.png" alt="enter image description here"></a></p>
  
  <p>where
  g stands for the logistic function
  g' stands for g's derivative
  w stands for weight
  hw(x) represents the logistic regression hypothesis</p>
  
  <p><strong>The other method:</strong></p>
  
  <p>Source (Paper authored by Charles Elkan): Logistic Regression and Stochastic Gradient Training.</p>
  
  <p>can be found <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.3611&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer">here</a></p>
  
  <p><a href="https://i.sstatic.net/ZCpRd.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZCpRd.png" alt="enter image description here"></a></p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-07T18:58:20.323Z
Comments.Text: How do you like that AI book? It’s on my reading list but I’ve done a fair amount of machine learning, deep learning, and now some RL, so I didn’t know how much would be new info. Would be interested in your take on it.
Comments.UserDisplayName: Hanzy
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-05-08T07:17:57.833Z
Comments.Text: @Hanzy Hi, well to be honest, my strategy is to search the subject at hand over several books. Thus, I don’t go through a whole book. However, what I likes about it is the clarity and it was used by an AI veteran of https://machinelearningmastery.com.
Comments.UserDisplayName: GYSHIDO
Tags: <machine-learning><gradient-descent><logistic-regression>
Title: What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent
UserDisplayName: GYSHIDO
ThreadText: # What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent
  
  (Asked by: username_0 on 2019-05-07)
  
  Apologies for the lengthy title.
  My question is about the weight update rule for logistic regression using stochastic gradient descent.
  
  I have just started experimenting on Logistic Regression. I came across two weight update expressions and did not know which one is more accurate and why they are different.
  
  **The first Method:**
  
  Source: (Book) Artificial Intelligence: A Modern Approach by Norvig, Russell
  on page 726-727:
  using the L2 loss function:
  
  [![enter image description here](https://i.sstatic.net/QA8O9.png)](https://i.sstatic.net/QA8O9.png)
  
  [![enter image description here](https://i.sstatic.net/FwVLC.png)](https://i.sstatic.net/FwVLC.png)
  
  where
  g stands for the logistic function
  g' stands for g's derivative
  w stands for weight
  hw(x) represents the logistic regression hypothesis
  
  **The other method:**
  
  Source (Paper authored by Charles Elkan): Logistic Regression and Stochastic Gradient Training.
  
  can be found [here](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.3611&rep=rep1&type=pdf)
  
  [![enter image description here](https://i.sstatic.net/ZCpRd.png)](https://i.sstatic.net/ZCpRd.png)
  
  **Tags:** machine-learning, gradient-descent, logistic-regression
  > **Comments:**
  > **username_1**: How do you like that AI book? It’s on my reading list but I’ve done a fair amount of machine learning, deep learning, and now some RL, so I didn’t know how much would be new info. Would be interested in your take on it.  
  > **username_0**: @Hanzy Hi, well to be honest, my strategy is to search the subject at hand over several books. Thus, I don’t go through a whole book. However, what I likes about it is the clarity and it was used by an AI veteran of https://machinelearningmastery.com.  
  
  ---
  
  ## 0 answers
  
  