Answers.Body: <p>I assume by Chained Linear Regression (CLR) you mean something like <span class="math-container">$f_1(f_2(x)) = y$</span>, where the functions <span class="math-container">$f_i$</span> would be your Linear Regression models. In that case, CLR cannot model non-linear dependencies, as <strong>the composition of two linear functions is itself linear</strong>.
  For the same reason, your second point is correct.</p>
  <p>And to answer your question: A multilayer perceptron (i.e. feedforward NN with multiple layers) consists of multiple layers and each layer is just a matrix multiplication. In CLR, each of your functions <span class="math-container">$f_i$</span> is also just a matrix multiplication (the parameters of these matrices are optimized during training). So in that sense, a multilayer perceptron without non-linearities and chained linear regression are the same thing. However, the data may flow a bit differently through a chained linear regression, so in the exact implementation, the models may still differ a bit, just not in their ability to model non-linear dependencies.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-19T22:20:06.470Z
Answers.Text: "So in that sense, a multilayer perceptron without non-linearities " by non-linearities you meant without activation function, if this is correct can I simplify that CLR + activation function = feedforward NN ?
Answers.UserDisplayName: Klembajnsztajn
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-20T01:08:54.107Z
Answers.Text: "by [without] non-linearities you meant without activation function", exactly! 
  To your question: "CLR + activation function(s) = feedforward NN + activation function(s)" If you say it like this, then it's correct. Note though that both feedforward NN and CLR are families of models: A feedforward neural network is everyneural network that doesn't contain recurrent connections (neurons feeding back information to an earlier layer) but it can take many forms. It only describes that information flows onyl one way. Multilayer perceptron is a better description of what you mean here I guess.
Answers.UserDisplayName: Chillston
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-19T18:33:56.340Z
Answers.LastActivityDate: 2024-01-19T18:33:56.340Z
Answers.UserDisplayName: Chillston
Body: <p>I am trying to understand the difference between feedforward NN and chained linear regression models, if and why they can model nonlinear functions.</p>
  <ul>
  <li>both are able to model non-linear dependencies</li>
  <li>feedforward NN without activation functions cannot model nonlinear dependecies and number of layers does not matter in this context</li>
  <li>chained regression models can model nonlinear dependencies</li>
  </ul>
  <p>I am confused because above statements comes from chatgpt and I am not sure if all are correct I believe they are mutually contradictory, because from what I know feedforward NN without activation function can be cosidered as chained linear regression models.</p>
  
Tags: <regression><models><feedforward-neural-networks>
Title: chained linear regression models vs feed forward NN
UserDisplayName: Klembajnsztajn
ThreadText: # chained linear regression models vs feed forward NN
  
  (Asked by: username_0 on 2024-01-19)
  
  I am trying to understand the difference between feedforward NN and chained linear regression models, if and why they can model nonlinear functions.
  
  * both are able to model non-linear dependencies
  * feedforward NN without activation functions cannot model nonlinear dependecies and number of layers does not matter in this context
  * chained regression models can model nonlinear dependencies
  
  I am confused because above statements comes from chatgpt and I am not sure if all are correct I believe they are mutually contradictory, because from what I know feedforward NN without activation function can be cosidered as chained linear regression models.
  
  **Tags:** regression, models, feedforward-neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-01-19. Score: 2 | Accepted answer
  
  I assume by Chained Linear Regression (CLR) you mean something like $f\_1(f\_2(x)) = y$, where the functions $f\_i$ would be your Linear Regression models. In that case, CLR cannot model non-linear dependencies, as **the composition of two linear functions is itself linear**.
  For the same reason, your second point is correct.
  
  And to answer your question: A multilayer perceptron (i.e. feedforward NN with multiple layers) consists of multiple layers and each layer is just a matrix multiplication. In CLR, each of your functions $f\_i$ is also just a matrix multiplication (the parameters of these matrices are optimized during training). So in that sense, a multilayer perceptron without non-linearities and chained linear regression are the same thing. However, the data may flow a bit differently through a chained linear regression, so in the exact implementation, the models may still differ a bit, just not in their ability to model non-linear dependencies.
  
  > **Comments:**
  > **username_0**: "So in that sense, a multilayer perceptron without non-linearities " by non-linearities you meant without activation function, if this is correct can I simplify that CLR + activation function = feedforward NN ?  
  > **username_1**: "by [without] non-linearities you meant without activation function", exactly!  To your question: "CLR + activation function(s) = feedforward NN + activation function(s)" If you say it like this, then it's correct. Note though that both feedforward NN and CLR are families of models: A feedforward neural network is everyneural network that doesn't contain recurrent connections (neurons feeding back information to an earlier layer) but it can take many forms. It only describes that information flows onyl one way. Multilayer perceptron is a better description of what you mean here I guess.  
  
  ---
  
  