<p>Apologies for the lengthy title.
My question is about the weight update rule for logistic regression using stochastic gradient descent.</p>

<p>I have just started experimenting on Logistic Regression. I came across two weight update expressions and did not know which one is more accurate and why they are different.</p>

<p><strong>The first Method:</strong></p>

<p>Source: (Book) Artificial Intelligence: A Modern Approach by Norvig, Russell
 on page 726-727:
using the L2 loss function:</p>

<p><a href="https://i.sstatic.net/QA8O9.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/QA8O9.png" alt="enter image description here"></a></p>

<p><a href="https://i.sstatic.net/FwVLC.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/FwVLC.png" alt="enter image description here"></a></p>

<p>where
g stands for the logistic function
g' stands for g's derivative
w stands for weight
hw(x) represents the logistic regression hypothesis</p>

<p><strong>The other method:</strong></p>

<p>Source (Paper authored by Charles Elkan): Logistic Regression and Stochastic Gradient Training.</p>

<p>can be found <a href="http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.3611&amp;rep=rep1&amp;type=pdf" rel="nofollow noreferrer">here</a></p>

<p><a href="https://i.sstatic.net/ZCpRd.png" rel="nofollow noreferrer"><img src="https://i.sstatic.net/ZCpRd.png" alt="enter image description here"></a></p>

CC BY-SA 4.0
2019-05-07T18:58:20.323Z
How do you like that AI book? It’s on my reading list but I’ve done a fair amount of machine learning, deep learning, and now some RL, so I didn’t know how much would be new info. Would be interested in your take on it.
Hanzy
CC BY-SA 4.0
2019-05-08T07:17:57.833Z
@Hanzy Hi, well to be honest, my strategy is to search the subject at hand over several books. Thus, I don’t go through a whole book. However, what I likes about it is the clarity and it was used by an AI veteran of https://machinelearningmastery.com.
GYSHIDO
<machine-learning><gradient-descent><logistic-regression>
What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent
GYSHIDO
# What is the right formula for weight update rule in Logistic Regression using stochastic gradient descent

(Asked by: username_0 on 2019-05-07)

Apologies for the lengthy title.
My question is about the weight update rule for logistic regression using stochastic gradient descent.

I have just started experimenting on Logistic Regression. I came across two weight update expressions and did not know which one is more accurate and why they are different.

**The first Method:**

Source: (Book) Artificial Intelligence: A Modern Approach by Norvig, Russell
on page 726-727:
using the L2 loss function:

[![enter image description here](https://i.sstatic.net/QA8O9.png)](https://i.sstatic.net/QA8O9.png)

[![enter image description here](https://i.sstatic.net/FwVLC.png)](https://i.sstatic.net/FwVLC.png)

where
g stands for the logistic function
g' stands for g's derivative
w stands for weight
hw(x) represents the logistic regression hypothesis

**The other method:**

Source (Paper authored by Charles Elkan): Logistic Regression and Stochastic Gradient Training.

can be found [here](http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.169.3611&rep=rep1&type=pdf)

[![enter image description here](https://i.sstatic.net/ZCpRd.png)](https://i.sstatic.net/ZCpRd.png)

**Tags:** machine-learning, gradient-descent, logistic-regression

> **Comments:**
> **username_1**: How do you like that AI book? It’s on my reading list but I’ve done a fair amount of machine learning, deep learning, and now some RL, so I didn’t know how much would be new info. Would be interested in your take on it.  
> **username_0**: @username_1 Hi, well to be honest, my strategy is to search the subject at hand over several books. Thus, I don’t go through a whole book. However, what I likes about it is the clarity and it was used by an AI veteran of https://machinelearningmastery.com.  

---

## 0 answers

