Answers.Body: <p>Though conceptually they're very similar, your own reference points out their essential difference:</p>
  <blockquote>
  <p>This update rule minimizes <span class="math-container">$E$</span>, the square of the error, and is in fact the stochastic gradient descent update for linear regression.</p>
  </blockquote>
  <p>Thus Adaline usually as a delta training rule for a (single) linear output layer in a ANN updates the model's weights incrementally based on individual training samples, which can lead to faster convergence of its error function and is suitable for online learning scenarios where data is continuously streaming. While the traditional OLS closed-form analytical approach for multiple linear regression in statistics requires all sample in a single batch.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-04T23:42:38.013Z
Answers.Text: Thanks!. I have other doubt about your answer: Does the fact that traditional OLS uses a single batch, and Adaline based on individual training samples, affect the accuracy or the final result? or should both produce similar results as well?
  
  please explain more about the batch?
Answers.UserDisplayName: will The J
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-04T23:47:15.370Z
Answers.Text: Of course OLS is more accurate once you gather all data inputs provided it has its closed form solution relative to SGD and even full GD. It's well known GD could be very slow and may converge to local minimum while OLS attains global minimum of squared error.
Answers.UserDisplayName: cinch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-04T23:53:20.307Z
Answers.Text: Thanks!. Does this mean that although Adaline can be used for multiple linear regression, it runs the risk of getting stuck in a local minimum during training? while OLS doesn't run this risk, and can be more accurate in some cases? I am correct?
Answers.UserDisplayName: will The J
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-05T00:17:01.577Z
Answers.Text: Yes that’s certainly a reasonable conclusion and historically the backpropagation algo based upon gradient descent for MLPs was inspired by the Adaline SGD, not OLS.
Answers.UserDisplayName: cinch
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-11-04T23:36:33.360Z
Answers.LastActivityDate: 2023-11-04T23:36:33.360Z
Answers.UserDisplayName: cinch
Body: <p><a href="https://en.wikipedia.org/wiki/ADALINE" rel="nofollow noreferrer">https://en.wikipedia.org/wiki/ADALINE</a></p>
  <p>Can Adaline(Adaptive Linear Neuron) be used to do a multiple linear regression being equivalent to the least squares method?</p>
  
Tags: <linear-regression>
Title: Can Adaline do multiple linear regression being equivalent to the least squares method?
UserDisplayName: will The J
ThreadText: # Can Adaline do multiple linear regression being equivalent to the least squares method?
  
  (Asked by: username_0 on 2023-11-04)
  
  <https://en.wikipedia.org/wiki/ADALINE>
  
  Can Adaline(Adaptive Linear Neuron) be used to do a multiple linear regression being equivalent to the least squares method?
  
  **Tags:** linear-regression
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2023-11-04. Score: 2
  
  Though conceptually they're very similar, your own reference points out their essential difference:
  
  > This update rule minimizes $E$, the square of the error, and is in fact the stochastic gradient descent update for linear regression.
  
  Thus Adaline usually as a delta training rule for a (single) linear output layer in a ANN updates the model's weights incrementally based on individual training samples, which can lead to faster convergence of its error function and is suitable for online learning scenarios where data is continuously streaming. While the traditional OLS closed-form analytical approach for multiple linear regression in statistics requires all sample in a single batch.
  
  > **Comments:**
  > **username_0**: Thanks!. I have other doubt about your answer: Does the fact that traditional OLS uses a single batch, and Adaline based on individual training samples, affect the accuracy or the final result? or should both produce similar results as well?  please explain more about the batch?  
  > **username_1**: Of course OLS is more accurate once you gather all data inputs provided it has its closed form solution relative to SGD and even full GD. It's well known GD could be very slow and may converge to local minimum while OLS attains global minimum of squared error.  
  > **username_0**: Thanks!. Does this mean that although Adaline can be used for multiple linear regression, it runs the risk of getting stuck in a local minimum during training? while OLS doesn't run this risk, and can be more accurate in some cases? I am correct?  
  > **username_1**: Yes that’s certainly a reasonable conclusion and historically the backpropagation algo based upon gradient descent for MLPs was inspired by the Adaline SGD, not OLS.  
  
  ---
  
  