Answers.Body: <p>Yes, predicting deltas is often easier than predicting values directly. The most obvious way to perform this task would be to use a simple MLP with MSE loss. If the deltas are discrete, and there are only 9 possible values, a multi-label cross entropy loss would probably perform better than using MSE.</p>
  <p>Depending on what the relationships between the numbers are, you might want to feed the last W numbers to the neural network, so that it can learn to use context. To do that, just make your input dimension W, and have your output dimension be 1. If you need a very long context window for good prediction, you should consider using an RNN.</p>
  <p>One thing that would help to improve training is that the scale of the inputs shouldn't be too different (e.g. some inputs being ~10^5 and some being ~10^0). If you can rescale your inputs to always be close to 0, that's helpful. For example, if the rule which relates numbers to one another is +3, +5, +3, +5, ... you could always normalize the first input to be 0, and un-normalize after you get your neural network output, without any loss of information.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-09-27T23:47:18.017Z
Answers.LastActivityDate: 2022-09-27T23:47:18.017Z
Answers.UserDisplayName: mdc
Body: <p>I have a list of increasing numbers with no duplicates for example : 3,6,11 and so on. Difference or deltas between these numbers such as in above case : 3, 3, 5 are frequent and with greatest difference being 9. Can a neural network be used to predict next 3 deltas for next 3 numbers. Which type of neural network will be useful and what kind of limitations will there be ?
  <br>
  Note : List comprises of millions of numbers.
  These numbers and deltas are not random.</p>
  
Tags: <neural-networks>
Title: Can neural network be used to predict deltas between numbers?
UserDisplayName: Akash
ThreadText: # Can neural network be used to predict deltas between numbers?
  
  (Asked by: username_0 on 2022-09-27)
  
  I have a list of increasing numbers with no duplicates for example : 3,6,11 and so on. Difference or deltas between these numbers such as in above case : 3, 3, 5 are frequent and with greatest difference being 9. Can a neural network be used to predict next 3 deltas for next 3 numbers. Which type of neural network will be useful and what kind of limitations will there be ?
    
  Note : List comprises of millions of numbers.
  These numbers and deltas are not random.
  
  **Tags:** neural-networks
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-09-27. Score: 3
  
  Yes, predicting deltas is often easier than predicting values directly. The most obvious way to perform this task would be to use a simple MLP with MSE loss. If the deltas are discrete, and there are only 9 possible values, a multi-label cross entropy loss would probably perform better than using MSE.
  
  Depending on what the relationships between the numbers are, you might want to feed the last W numbers to the neural network, so that it can learn to use context. To do that, just make your input dimension W, and have your output dimension be 1. If you need a very long context window for good prediction, you should consider using an RNN.
  
  One thing that would help to improve training is that the scale of the inputs shouldn't be too different (e.g. some inputs being ~10^5 and some being ~10^0). If you can rescale your inputs to always be close to 0, that's helpful. For example, if the rule which relates numbers to one another is +3, +5, +3, +5, ... you could always normalize the first input to be 0, and un-normalize after you get your neural network output, without any loss of information.
  
  
  ---
  
  