Answers.Body: <p>As DKDK said, Indeed one could fit both linear and exponential function and see which one has smaller residual, without using any complex AI.</p>
  <p>But OTOH this could be a great toy-problem for learning about neural networks. You could have a network with these parts:</p>
  <ul>
  <li>A network with a final sigmoid activation, which predicts whether the function is linear or not. Let's call its prediction <code>C</code> for &quot;class&quot;.</li>
  <li>An additional &quot;data path&quot; within the same network, which predicts <strong>linear</strong> functions. Let's call its prediction <code>L</code> for &quot;linear&quot;.</li>
  <li>An additional &quot;data path&quot; within the same network, which predicts <strong>exponential</strong> functions. Let's call its prediction <code>E</code> for &quot;exponential&quot;.</li>
  <li>There is just a single input but two outputs. The first output is <code>C</code> and the second one is <code>C * L + (1 - C) * E</code>.</li>
  </ul>
  <p>Then you can train the whole network end-to-end, but this model could output a mix of 20% linear and 80% exponential values. If this is a problem then I'd use three separate networks as DKDK suggested.</p>
  <p>Edit: Now that I read the question again I noticed that all your linear functions have a negative slope? Well surely they will be easy to identify by comparing the first and last element of the list. Then this is just a normal regression problem.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-05T22:07:48.103Z
Answers.LastActivityDate: 2022-01-06T07:02:09.407Z
Answers.LastEditDate: 2022-01-06T07:02:09.407Z
Answers.UserDisplayName: NikoNyrh
Answers.Body: <p>How about dividing the problem?</p>
  <p>You can first train a classification model that predicts the type of function (linear or exponential).</p>
  <p>Then you can use your seperately trained nn depending on the classification output.</p>
  <p>P.S.</p>
  <p>I'm not sure why you would use a neural network for this problem. Fitting a linear/exponential function seems to be a relatively simple problem that can be achieved using traditional ML approaches.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-01-05T02:09:37.793Z
Answers.LastActivityDate: 2022-01-05T02:09:37.793Z
Answers.UserDisplayName: DKDK
Body: <p>I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict are the following:</p>
  <ol>
  <li>a linear function with negative slope</li>
  <li>a significantly steeper exponential function.</li>
  </ol>
  <p>Example inputs for each would look like</p>
  <ol>
  <li>[10, 9, 8, 7]</li>
  <li>[11, 1, 0.1, 0]</li>
  </ol>
  <p>Sometimes my neural network predicts well, such as the following:
  <a href="https://i.sstatic.net/wzzK4.jpg" rel="nofollow noreferrer">Linear</a></p>
  <p>Other times my neural network completely messes up and predicts the exponential decay instead:
  <a href="https://i.sstatic.net/9FEzm.jpg" rel="nofollow noreferrer">Poor Prediction on Linear</a></p>
  <p>As you can imagine, this makes the final result a very poor one.</p>
  <p>I'm using a NN architecture of <a href="https://towardsdatascience.com/neural-collaborative-filtering-96cef1009401" rel="nofollow noreferrer">NCF</a> because I'm setting up from a matrix setting. That is, the arrays I have generated come from a noisy matrix setting of low rank.</p>
  <p>Any suggestions to improve the result? When I only use one function (e.g. just linear or just exponential) I get superb results. However, mixing both those functions together randomly causes my neural network to perform poorly, as described in my question.</p>
  
Tags: <neural-networks><machine-learning><deep-learning><pytorch><collaborative-filtering>
Title: How to make NN distinguish between two types of functions (data)?
UserDisplayName: Galen BlueTalon
ThreadText: # How to make NN distinguish between two types of functions (data)?
  
  (Asked by: username_0 on 2022-01-02)
  
  I have a neural network which is trying to predict two types of functions in a noisy setting. The input is an array, and the output is also an array. The two types of functions I am trying to predict are the following:
  
  1. a linear function with negative slope
  2. a significantly steeper exponential function.
  
  Example inputs for each would look like
  
  1. [10, 9, 8, 7]
  2. [11, 1, 0.1, 0]
  
  Sometimes my neural network predicts well, such as the following:
  [Linear](https://i.sstatic.net/wzzK4.jpg)
  
  Other times my neural network completely messes up and predicts the exponential decay instead:
  [Poor Prediction on Linear](https://i.sstatic.net/9FEzm.jpg)
  
  As you can imagine, this makes the final result a very poor one.
  
  I'm using a NN architecture of [NCF](https://towardsdatascience.com/neural-collaborative-filtering-96cef1009401) because I'm setting up from a matrix setting. That is, the arrays I have generated come from a noisy matrix setting of low rank.
  
  Any suggestions to improve the result? When I only use one function (e.g. just linear or just exponential) I get superb results. However, mixing both those functions together randomly causes my neural network to perform poorly, as described in my question.
  
  **Tags:** neural-networks, machine-learning, deep-learning, pytorch, collaborative-filtering
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2022-01-05. Score: 2 | Accepted answer
  
  How about dividing the problem?
  
  You can first train a classification model that predicts the type of function (linear or exponential).
  
  Then you can use your seperately trained nn depending on the classification output.
  
  P.S.
  
  I'm not sure why you would use a neural network for this problem. Fitting a linear/exponential function seems to be a relatively simple problem that can be achieved using traditional ML approaches.
  
  
  ---
  
  ### Answer by username_1 on 2022-01-05. Score: 1
  
  As DKDK said, Indeed one could fit both linear and exponential function and see which one has smaller residual, without using any complex AI.
  
  But OTOH this could be a great toy-problem for learning about neural networks. You could have a network with these parts:
  
  * A network with a final sigmoid activation, which predicts whether the function is linear or not. Let's call its prediction `C` for "class".
  * An additional "data path" within the same network, which predicts **linear** functions. Let's call its prediction `L` for "linear".
  * An additional "data path" within the same network, which predicts **exponential** functions. Let's call its prediction `E` for "exponential".
  * There is just a single input but two outputs. The first output is `C` and the second one is `C * L + (1 - C) * E`.
  
  Then you can train the whole network end-to-end, but this model could output a mix of 20% linear and 80% exponential values. If this is a problem then I'd use three separate networks as DKDK suggested.
  
  Edit: Now that I read the question again I noticed that all your linear functions have a negative slope? Well surely they will be easy to identify by comparing the first and last element of the list. Then this is just a normal regression problem.
  
  
  ---
  
  