Answers.Body: <p>There is a difference between adding more samples to the data (rows), and adding more features (columns). In this case we are talking about more features.</p>
  <p>If the function <span class="math-container">$f$</span> is trivial, feeding extra columns doesn't hurt but doesn't bring any benefits either. And the training is a bit slower, since extra gradients needs to be calculated. If it is non-trivial, it could be the case that this helps the network train faster and maybe you can even use a smaller network.</p>
  <p>It is common to pre-process data before feeding it to the network, for example transforming a time-feature into two season-features via <code>sin</code> and <code>cos</code> transformations.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-03T09:53:18.383Z
Answers.LastActivityDate: 2023-03-03T09:53:18.383Z
Answers.UserDisplayName: NikoNyrh
Answers.Body: <p>No it is not useless.</p>
  <p>The relationship may not be obvious, and having the data will allow the network to learn this  relationship.</p>
  <p>Further, even if  is obvious, networks are so sample inefficient that more (non-noisy) data is always helpful. In fact, the common practice is to train for hundreds of epochs on the same exact samples - because we can not learn quickly enough from seeing them only once.</p>
  <p>That said, there are some cases where data is harmful. For example, if we have an imbalanced dataset, adding more samples to exacerbate that imbalanced may be a bad idea.</p>
  <p>But in general, this added data will still be of use.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-03T11:52:53.717Z
Answers.Text: What’s wrong with imbalance?
Answers.UserDisplayName: Dave
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2023-03-02T14:55:07.780Z
Answers.LastActivityDate: 2023-03-02T22:45:58.877Z
Answers.LastEditDate: 2023-03-02T22:45:58.877Z
Answers.UserDisplayName: chessprogrammer
Body: <p>That is, if some of the inputs to a neural network can be calculated by a pre-determined function whose variables are other inputs, then are those specific inputs useless?</p>
  <p>For example, suppose there are three inputs, <span class="math-container">$x_1$</span>, <span class="math-container">$x_2$</span> and <span class="math-container">$x_3$</span>. If <span class="math-container">$x_3$</span> is determined by function <span class="math-container">$x_3=f(x_1,x_2)$</span>, then will <span class="math-container">$x_3$</span> be useless for training a neural network?</p>
  
Tags: <neural-networks><training><datasets><input-layer>
Title: Is data useless for a neural network if some inputs are derivatives of other inputs?
UserDisplayName: BlueSnake
ThreadText: # Is data useless for a neural network if some inputs are derivatives of other inputs?
  
  (Asked by: username_0 on 2023-03-02)
  
  That is, if some of the inputs to a neural network can be calculated by a pre-determined function whose variables are other inputs, then are those specific inputs useless?
  
  For example, suppose there are three inputs, $x\_1$, $x\_2$ and $x\_3$. If $x\_3$ is determined by function $x\_3=f(x\_1,x\_2)$, then will $x\_3$ be useless for training a neural network?
  
  **Tags:** neural-networks, training, datasets, input-layer
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_2 on 2023-03-02. Score: 2 | Accepted answer
  
  No it is not useless.
  
  The relationship may not be obvious, and having the data will allow the network to learn this relationship.
  
  Further, even if is obvious, networks are so sample inefficient that more (non-noisy) data is always helpful. In fact, the common practice is to train for hundreds of epochs on the same exact samples - because we can not learn quickly enough from seeing them only once.
  
  That said, there are some cases where data is harmful. For example, if we have an imbalanced dataset, adding more samples to exacerbate that imbalanced may be a bad idea.
  
  But in general, this added data will still be of use.
  
  > **Comments:**
  > **username_3**: What’s wrong with imbalance?  
  
  ---
  
  ### Answer by username_1 on 2023-03-03. Score: 0
  
  There is a difference between adding more samples to the data (rows), and adding more features (columns). In this case we are talking about more features.
  
  If the function $f$ is trivial, feeding extra columns doesn't hurt but doesn't bring any benefits either. And the training is a bit slower, since extra gradients needs to be calculated. If it is non-trivial, it could be the case that this helps the network train faster and maybe you can even use a smaller network.
  
  It is common to pre-process data before feeding it to the network, for example transforming a time-feature into two season-features via `sin` and `cos` transformations.
  
  
  ---
  
  