Answers.Body: <pre><code>X = np.array([1,2,3])
  Y = np.array([2,1,2])
  
  params = np.array([1, 0])
  
  def loss(y, yhat):
      return ((y - yhat)**2).mean()
  
  def model(x):
      return params[0] + params[1]*x
  
  def loss_grad(y, yhat, x):
      return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])
  
  lr = .1
  for _ in range(3):
      yhat = model(X)
      l = loss(Y, yhat)
      g = loss_grad(Y, yhat, X)
      params = params - lr*g
      print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')
  </code></pre>
  
  <p>outputs </p>
  
  <pre><code>thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666
  thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296
  thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354
  </code></pre>
  
  <p>I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-06-03T20:55:12.187Z
Answers.LastActivityDate: 2019-06-03T20:55:12.187Z
Answers.UserDisplayName: mshlis
Body: <blockquote>
    <p>Consider the following data with one input (x) and one output (y):<br>
    (x=1, y=2)<br>
    (x=2, y=1)<br>
    (x=3, y=2)<br>
    Apply linear regression on this data, using the hypothesis <span class="math-container">$h_Θ(x) = Θ_0 + Θ_1 x$</span>, where <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> represent the parameters to be learned. Considering the initial values <span class="math-container">$Θ_0$</span>= 1.0, and <span class="math-container">$Θ_1$</span> = 0.0, and learning rate 0.1, what will be the values of <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> after the first three iterations of Gradient Descent</p>
  </blockquote>
  
  <p>From least squares method I took the derivative with respect to <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new <span class="math-container">$Θ_0$</span> and <span class="math-container">$Θ_1$</span> values.  </p>
  
  <p>I am getting <span class="math-container">$Θ_0$</span> as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve</p>
  
Tags: <machine-learning><linear-regression>
Title: Calculating Parameter value Using Gradient Descent for Linear Regression Model
UserDisplayName: ten do
ThreadText: # Calculating Parameter value Using Gradient Descent for Linear Regression Model
  
  (Asked by: username_0 on 2019-06-03)
  
  > Consider the following data with one input (x) and one output (y):  
  > (x=1, y=2)  
  > (x=2, y=1)  
  > (x=3, y=2)  
  > Apply linear regression on this data, using the hypothesis $h\_Θ(x) = Θ\_0 + Θ\_1 x$, where $Θ\_0$ and $Θ\_1$ represent the parameters to be learned. Considering the initial values $Θ\_0$= 1.0, and $Θ\_1$ = 0.0, and learning rate 0.1, what will be the values of $Θ\_0$ and $Θ\_1$ after the first three iterations of Gradient Descent
  
  From least squares method I took the derivative with respect to $Θ\_0$ and $Θ\_1$ and plugged in the initial values to get the slope/intercept and multiplied it by the learning rate 0.1 to get the step size.The step size was used to calculate the new $Θ\_0$ and $Θ\_1$ values.
  
  I am getting $Θ\_0$ as 1.7821 when following the above. Please let me know if the approach followed and the solution correct or there is a better way to solve
  
  **Tags:** machine-learning, linear-regression
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2019-06-03. Score: 1 | Accepted answer
  
  X = np.array([1,2,3])
  Y = np.array([2,1,2])
  
  params = np.array([1, 0])
  
  def loss(y, yhat):
      return ((y - yhat)**2).mean()
  
  def model(x):
      return params[0] + params[1]*x
  
  def loss_grad(y, yhat, x):
      return np.array([(2*(yhat-y)).mean(), (2*(yhat-y)*x).mean()])
  
  lr = .1
  for _ in range(3):
      yhat = model(X)
      l = loss(Y, yhat)
      g = loss_grad(Y, yhat, X)
      params = params - lr*g
      print(f'thetas are now {params} with new loss of {loss(Y, yhat)}')
  
  outputs 
  thetas are now [1.13333333 0.26666667] with new loss of 0.6666666666666666
  thetas are now [1.13333333 0.23111111] with new loss of 0.2696296296296296
  thetas are now [1.14755556 0.22874074] with new loss of 0.262887242798354
  
  I double checked this with keras too, but in numpy i explicitly wrote the gradients, i reccomend double checking your gradient or arithmetic
  
  
  ---
  
  