<pre><code>X = np.array([1,2,3])
Y = np.array([2,1,2])

params = np.array([1, 0])

def loss(y, yhat):
    return ((y - yhat)**2).mean()

def model(x):
    return params[0] + params[1]*x

def loss_grad(y, yhat, x):
    return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])

lr = .1
for _ in range(3):
    yhat = model(X)
    l = loss(Y, yhat)
    g = loss_grad(Y, yhat, X)
    params = params - lr*g
    print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')
</code></pre>

<p>outputs </p>

<pre><code>thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666
thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296
thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354
</code></pre>

<p>I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic</p>

CC BY-SA 4.0
2019-06-03T20:55:12.187Z
2019-06-03T20:55:12.187Z
mshlis
<blockquote>
  <p>Consider the following data with one input (x) and one output (y):<br>
  (x=1, y=2)<br>
  (x=2, y=1)<br>
  (x=3, y=2)<br>
  Apply linear regression on this data, using the hypothesis <span class="math-container">$h_Θ(x) = Θ_0 + Θ_1 x$</span>, where <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> represent the parameters to be learned. Considering the initial values <span class="math-container">$Θ_0$</span>= 1.0, and <span class="math-container">$Θ_1$</span> = 0.0, and learning rate 0.1, what will be the values of <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> after the first three iterations of Gradient Descent</p>
</blockquote>

<p>From least squares method I took the derivative with respect to <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> values.  </p>

<p>I am getting <span class="math-container">$Θ_0$</span> as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve</p>

<machine-learning><linear-regression>
Calculating Parameter value Using Gradient Descent for Linear Regression Model
ten do
# Calculating Parameter value Using Gradient Descent for Linear Regression Model

(Asked by: username_0 on 2019-06-03)

> Consider the following data with one input (x) and one output (y):  
> (x=1, y=2)  
> (x=2, y=1)  
> (x=3, y=2)  
> Apply linear regression on this data, using the hypothesis $h\_Θ(x) = Θ\_0 + Θ\_1 x$, where $Θ\_0$ and $Θ\_1$ represent the parameters to be learned. Considering the initial values $Θ\_0$= 1.0, and $Θ\_1$ = 0.0, and learning rate 0.1, what will be the values of $Θ\_0$ and $Θ\_1$ after the first three iterations of Gradient Descent

From least squares method I took the derivative with respect to $Θ\_0$ and $Θ\_1$ and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new $Θ\_0$ and $Θ\_1$ values.

I am getting $Θ\_0$ as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve

**Tags:** machine-learning, linear-regression


---

## 1 answers

### Answer by username_1 on 2019-06-03. Score: 1 | Accepted answer

X = np.array([1,2,3])
Y = np.array([2,1,2])

params = np.array([1, 0])

def loss(y, yhat):
    return ((y - yhat)**2).mean()

def model(x):
    return params[0] + params[1]*x

def loss_grad(y, yhat, x):
    return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])

lr = .1
for _ in range(3):
    yhat = model(X)
    l = loss(Y, yhat)
    g = loss_grad(Y, yhat, X)
    params = params - lr*g
    print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')

outputs 
thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666
thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296
thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354

I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic


---

