Body: <p>I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this?</p>
  
  <p>The learn data only contains extreme situations where each row contains one and only one output value set to 100.
  The data to predict is expected to contain more nuances in the output values.
  All my attempts lead to very low accuracy as the predicted output sum is almost never a 100. Even when I try to normalize the predicted output, the predictions show very poor accuracy.</p>
  
  <p>Should I try to organize the data with 2 angles instead and deduct the 3rd angle as the remainder in a circle? How to normalize those 2 angles and how to make sure their sum will not exceed the maximum value making the 3rd angle negative?</p>
  
  <p>Illustration of learn data extract (4 input columns and 3 output columns).</p>
  
  <pre><code>0    1    2    3    100  0    0
  4    5    6    7    0    100  0
  8    9    0    1    0    0    100
  </code></pre>
  
  <p>Illustration of desired output predictions where each line sums as 100:</p>
  
  <pre><code>7    83   10
  39   12   49
  68   24   8
  28   72   0
  86   6    8
  32   49   19
  0    0    100
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-21T15:05:11.880Z
Comments.Text: I think Normalization is the route— what approach did you try?
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-21T15:20:28.480Z
Comments.Text: I tried several normalization approaches. One of them consisted in dividing each output value by the sum of the 3 and multiply by 100. Does this make sense?
Comments.UserDisplayName: luvzfootball
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-21T15:22:07.987Z
Comments.Text: Yes. That should be fine. What’s your loss?
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-21T15:24:08.790Z
Comments.Text: Thanks for the confirmation. I didn't even try to calculate the loss, the numbers were completely off. Maybe my issue is not with the normalization then? Maybe I should rethink the whole learning process?
Comments.UserDisplayName: luvzfootball
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-23T01:48:16.997Z
Comments.Text: If you don’t have a loss— how did you train?
Comments.UserDisplayName: mshlis
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-23T17:45:12.137Z
Comments.Text: I use a Jupyter MLPRegressor with a StandardScaler as explained [here](https://scikit-learn.org/stable/modules/neural_networks_supervised.html#tips-on-practical-use). I used various maximum number of iterations (1000-10000) and hidden layer sizes (50,50 or 200,200,200). I used different activation functions and stick to logistic. I recently got much better predictions by removing several less relevant input columns. Sorry, I didn't realize the loss is necessarily calculated. I get now `'MLPRegressor' has no attribute 'loss_curve_'` error when I try to plot. I'll look for a solution for that.
Comments.UserDisplayName: luvzfootball
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-07-25T13:53:38.027Z
Comments.Text: The error I had regarding the loss was due to a wrong copy/paste. I get a loss of 19 (starting around 1600) with 10000 iterations, layer sizes (200,200,200) and a relu activation function. Predictions are not perfect but it is a lot better.
Comments.UserDisplayName: luvzfootball
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-08-03T10:47:54.163Z
Comments.Text: @mshlis , my results are not very convincing. I wonder whether a multi output activation function to take into account this proportion constraint in the last neural network layer could help. It seems that the "softmax" activation function could be an answer to this. Though, I'm not sure I can easily use such a solution with jupyter.
Comments.UserDisplayName: luvzfootball
Tags: <neural-networks><machine-learning><multilayer-perceptrons>
Title: How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?
UserDisplayName: luvzfootball
ThreadText: # How can an ANN efficiently predict multiple numbers with fixed sum (in other words, proportions)?
  
  (Asked by: username_0 on 2019-07-21)
  
  I need a neural network (or any other solution) to predict 3 values which sum equals a fixed number (100). This will help me calculate proportions. Which is the most efficient way to do this?
  
  The learn data only contains extreme situations where each row contains one and only one output value set to 100.
  The data to predict is expected to contain more nuances in the output values.
  All my attempts lead to very low accuracy as the predicted output sum is almost never a 100. Even when I try to normalize the predicted output, the predictions show very poor accuracy.
  
  Should I try to organize the data with 2 angles instead and deduct the 3rd angle as the remainder in a circle? How to normalize those 2 angles and how to make sure their sum will not exceed the maximum value making the 3rd angle negative?
  
  Illustration of learn data extract (4 input columns and 3 output columns).
  
  ```
  0    1    2    3    100  0    0
  4    5    6    7    0    100  0
  8    9    0    1    0    0    100
  
  ```
  
  Illustration of desired output predictions where each line sums as 100:
  
  ```
  7    83   10
  39   12   49
  68   24   8
  28   72   0
  86   6    8
  32   49   19
  0    0    100
  
  ```
  
  **Tags:** neural-networks, machine-learning, multilayer-perceptrons
  
  > **Comments:**
  > **username_1**: I think Normalization is the route— what approach did you try?  
  > **username_0**: I tried several normalization approaches. One of them consisted in dividing each output value by the sum of the 3 and multiply by 100. Does this make sense?  
  > **username_1**: Yes. That should be fine. What’s your loss?  
  > **username_0**: Thanks for the confirmation. I didn't even try to calculate the loss, the numbers were completely off. Maybe my issue is not with the normalization then? Maybe I should rethink the whole learning process?  
  > **username_1**: If you don’t have a loss— how did you train?  
  > **username_0**: I use a Jupyter MLPRegressor with a StandardScaler as explained [here](https://scikit-learn.org/stable/modules/neural_networks_supervised.html#tips-on-practical-use). I used various maximum number of iterations (1000-10000) and hidden layer sizes (50,50 or 200,200,200). I used different activation functions and stick to logistic. I recently got much better predictions by removing several less relevant input columns. Sorry, I didn't realize the loss is necessarily calculated. I get now `'MLPRegressor' has no attribute 'loss_curve_'` error when I try to plot. I'll look for a solution for that.  
  > **username_0**: The error I had regarding the loss was due to a wrong copy/paste. I get a loss of 19 (starting around 1600) with 10000 iterations, layer sizes (200,200,200) and a relu activation function. Predictions are not perfect but it is a lot better.  
  > **username_0**: @username_1 , my results are not very convincing. I wonder whether a multi output activation function to take into account this proportion constraint in the last neural network layer could help. It seems that the "softmax" activation function could be an answer to this. Though, I'm not sure I can easily use such a solution with jupyter.  
  
  ---
  
  ## 0 answers
  
  