Answers.Body: <p>Yes, however, most likely not at your scale</p>
  <p>you can read more here <a href="https://arxiv.org/abs/2001.08361" rel="nofollow noreferrer">https://arxiv.org/abs/2001.08361</a> but in poor words, they show that the scaling of the parameters and the data are not orthogonal, and that smaller models, on bigger datasets, have worse performances than trained on smaller dataset</p>
  <p>It's like they get overwhelmed of informations, and instead of becoming good at something, they get mediocre over everything (consider that this applies to LLMs, where they have to predict the next token, thus if you don't have enough memory to recall given a context what comes next, then you will produce a uniform distribution over the next token, getting terrible performances)</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-15T17:26:42.610Z
Answers.LastActivityDate: 2024-01-15T17:26:42.610Z
Answers.UserDisplayName: Alberto
Answers.Body: <p>If your data contains a lot of near repeated samples, it’s quite possible too many training data can hurt, assuming you are training for 1 epoch. This is the same phenomenon as overfitting when you train on the same dataset multiple epochs.</p>
  <p>Another reason to see degradation is if your increased training data is of lower quality.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-01-15T16:12:22.763Z
Answers.LastActivityDate: 2024-01-15T16:12:22.763Z
Answers.UserDisplayName: John Jiang
Body: <p>I have to detect objects in an image. I want to use a neural network for this (yolov8).
  Since my objects are stacked, most of them are partially hidden and only front and side is visible. My dataset for training contains 50% images where the objects are fully visible and 50% where the objects are stacked.
  In reality, only the top object of the stack is fully visible and there are 4-5 objects per stack, so maybe 25% of the detecable objects are fully visible and 75% are partially hidden.</p>
  <p>Can it have a negative impact if I use all of the training data? So should I use only the stacked-images for training or is it always a good idea to use as much trainingdata as possible?
  Maybe the model can find some useful informations of the objects in the fully-visible-images. Or maybe too much fully visible objects are misleading?</p>
  <p>Since the trainingdata isn't labeled yet I can't test both options. And I don't want to waste too much time in labeling useless data.</p>
  <p>What are your thoughts on this?
  Thanks!</p>
  
Tags: <neural-networks><training-datasets><data-labelling>
Title: Can too much trainingdata have a negative impact?
UserDisplayName: Ef Ge
ThreadText: # Can too much trainingdata have a negative impact?
  
  (Asked by: username_0 on 2024-01-15)
  
  I have to detect objects in an image. I want to use a neural network for this (yolov8).
  Since my objects are stacked, most of them are partially hidden and only front and side is visible. My dataset for training contains 50% images where the objects are fully visible and 50% where the objects are stacked.
  In reality, only the top object of the stack is fully visible and there are 4-5 objects per stack, so maybe 25% of the detecable objects are fully visible and 75% are partially hidden.
  
  Can it have a negative impact if I use all of the training data? So should I use only the stacked-images for training or is it always a good idea to use as much trainingdata as possible?
  Maybe the model can find some useful informations of the objects in the fully-visible-images. Or maybe too much fully visible objects are misleading?
  
  Since the trainingdata isn't labeled yet I can't test both options. And I don't want to waste too much time in labeling useless data.
  
  What are your thoughts on this?
  Thanks!
  
  **Tags:** neural-networks, training-datasets, data-labelling
  
  
  ---
  
  ## 2 answers
  
  ### Answer by username_1 on 2024-01-15. Score: 1
  
  Yes, however, most likely not at your scale
  
  you can read more here <https://arxiv.org/abs/2001.08361> but in poor words, they show that the scaling of the parameters and the data are not orthogonal, and that smaller models, on bigger datasets, have worse performances than trained on smaller dataset
  
  It's like they get overwhelmed of informations, and instead of becoming good at something, they get mediocre over everything (consider that this applies to LLMs, where they have to predict the next token, thus if you don't have enough memory to recall given a context what comes next, then you will produce a uniform distribution over the next token, getting terrible performances)
  
  
  ---
  
  ### Answer by username_2 on 2024-01-15. Score: 0
  
  If your data contains a lot of near repeated samples, it’s quite possible too many training data can hurt, assuming you are training for 1 epoch. This is the same phenomenon as overfitting when you train on the same dataset multiple epochs.
  
  Another reason to see degradation is if your increased training data is of lower quality.
  
  
  ---
  
  