Answers.Body: <blockquote>
    <p>For example, from among house size, lot size, age of house and asking price, what formula best predicts selling price?</p>
  </blockquote>
  
  <p>There is no general formula for this. Search for neural network regression and you can get started. The AI technique or any prediction algorithm in general will learn a function that maps from the input feature vector <span class="math-container">$(x_1, ...,x_n)$</span>, where each of the element in the vector is a measurement on the <span class="math-container">$\text{predictors/independent variables/regressors}$</span> to the <span class="math-container">$\text{variable of interest/dependent variables}$</span> i.e. <span class="math-container">$\text{selling price}$</span></p>
  
  <blockquote>
    <p>I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.</p>
  </blockquote>
  
  <p>The statement is incorrect. In fact the opposite is true. CNNs are known for modeling non-linear relationships. Examples are the highly successful image classification CNN architectures like Inception, ResNet, etc.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-04T20:22:51.357Z
Answers.Text: Before I go to modeling real-world data, I was trying to see if I could make a network that gave good results on a known, non-linear relationship such as the one between length of sides of a triangle to length of hypotenuse.   Is a sigmoid the right activation function to use in this context?   This is what i used but the results after training the network were, well, just wrong.
Answers.UserDisplayName: GGizmos
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-04T20:59:02.057Z
Answers.Text: You have a regression problem at hand. I would suggest you to use activation functions that are more suited for those for example leaky relu. Sigmoid is better for classification tasks.
Answers.UserDisplayName: naive
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-04T18:44:27.400Z
Answers.LastActivityDate: 2019-11-05T09:24:13.373Z
Answers.LastEditDate: 2019-11-05T09:24:13.373Z
Answers.UserDisplayName: naive
Answers.Body: <p>You are mixing up lots of things here. Specifically, you seem to be lacking a basic understanding of artificial neural networks and what they can do (e.g. what type of articifial neural networks are linear classifiers/regressors and which can model non-linear relationships). </p>
  
  <p>Therefore, I'd take a step back and start with understanding the basics of AI. The go-to book for that is 'Artificial Intelligence: A Modern Approach' by Russel and Norvig. It might be a slower (and more theoretical) start but IMO that is the right approach to actually understand what you are doing. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-11-04T19:34:31.773Z
Answers.LastActivityDate: 2019-11-04T19:34:31.773Z
Answers.UserDisplayName: Jonathan
Body: <p>I am interested in exploring whether AI techniques can derive hidden patterns of relationships in a data set.  For example, from among house size, lot size, age of house and asking price, what formula best predicts selling price?</p>
  <p>In explorations around how this might be done, I tried to use a neural network to solve for a predictable relationship between two variables to predict a third, so I trained my neural network with inputs consisting of the length of two sides of a triangle, and the result being the length of the hypotenuse.  It couldn't get it to work.</p>
  <p>I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.</p>
  <p>If that is true, I wonder if there is some other AI technique that could 'derive' a network modeling the Pythagorean theorem from a training data set with better results than a normal neural network?</p>
  
Tags: <neural-networks><machine-learning><algorithm-request><non-linear-regression>
Title: Which AI technique is best suited to discovering non-linear relationships in data?
UserDisplayName: GGizmos
ThreadText: # Which AI technique is best suited to discovering non-linear relationships in data?
  
  (Asked by: username_0 on 2019-11-04)
  
  I am interested in exploring whether AI techniques can derive hidden patterns of relationships in a data set. For example, from among house size, lot size, age of house and asking price, what formula best predicts selling price?
  
  In explorations around how this might be done, I tried to use a neural network to solve for a predictable relationship between two variables to predict a third, so I trained my neural network with inputs consisting of the length of two sides of a triangle, and the result being the length of the hypotenuse. It couldn't get it to work.
  
  I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.
  
  If that is true, I wonder if there is some other AI technique that could 'derive' a network modeling the Pythagorean theorem from a training data set with better results than a normal neural network?
  
  **Tags:** neural-networks, machine-learning, algorithm-request, non-linear-regression
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2019-11-04. Score: 2 | Accepted answer
  
  > For example, from among house size, lot size, age of house and asking price, what formula best predicts selling price?
  
  There is no general formula for this. Search for neural network regression and you can get started. The AI technique or any prediction algorithm in general will learn a function that maps from the input feature vector $(x\_1, ...,x\_n)$, where each of the element in the vector is a measurement on the $\text{predictors/independent variables/regressors}$ to the $\text{variable of interest/dependent variables}$ i.e. $\text{selling price}$
  
  > I was told by somebody who understands all this better than me that the reason it failed is because conventional neural networks are not good at modeling non-linear relationships.
  
  The statement is incorrect. In fact the opposite is true. CNNs are known for modeling non-linear relationships. Examples are the highly successful image classification CNN architectures like Inception, ResNet, etc.
  
  > **Comments:**
  > **username_0**: Before I go to modeling real-world data, I was trying to see if I could make a network that gave good results on a known, non-linear relationship such as the one between length of sides of a triangle to length of hypotenuse.   Is a sigmoid the right activation function to use in this context?   This is what i used but the results after training the network were, well, just wrong.  
  > **username_1**: You have a regression problem at hand. I would suggest you to use activation functions that are more suited for those for example leaky relu. Sigmoid is better for classification tasks.  
  
  ---
  
  ### Answer by username_2 on 2019-11-04. Score: 3
  
  You are mixing up lots of things here. Specifically, you seem to be lacking a basic understanding of artificial neural networks and what they can do (e.g. what type of articifial neural networks are linear classifiers/regressors and which can model non-linear relationships).
  
  Therefore, I'd take a step back and start with understanding the basics of AI. The go-to book for that is 'Artificial Intelligence: A Modern Approach' by Russel and Norvig. It might be a slower (and more theoretical) start but IMO that is the right approach to actually understand what you are doing.
  
  
  ---
  
  