Answers.Body: <p>He likely found this to a be a best practice to avoid over fitting, with a small data set if you only use small and easy to learn (less words -> less degrees of freedom) sequences then you open your model to the risk of over fitting that data set where as on a large data set that has alot more total information you can train on small sequences without being at risk of over fitting because although the smaller sequences will be easier to learn the variance of sequences will be much higher. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2019-10-30T18:55:01.937Z
Answers.LastActivityDate: 2019-10-30T18:55:01.937Z
Answers.UserDisplayName: nickw
Body: <p>In the deep learning specialization course by Andrew Ng, in the video <a href="https://www.coursera.org/lecture/nlp-sequence-models/negative-sampling-Iwx0e" rel="nofollow noreferrer">Sequence Models</a> (minute 4:13), he says that in negative sampling we have to choose a sample of words from the corpus to train rather than choosing the whole corpus. But he said that, for smaller datasets, we need a bigger number of samples, for example, 5-20, and, for larger datasets, we need a smaller sample, for example, 2-5. By sample, I am referring to the number of words along with the target word we have taken to train the model.</p>
  
  <p>Why do small datasets require more samples, while big datasets require fewer samples?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-30T09:37:41.303Z
Comments.Text: I think your title is a bit misleading. The way I read it was that you need to have more samples in small datasets, while you are referring to the negative samples in negative sampling (not dataset samples). Also your tags seem a bit irrelevant, especially the one about CNNs. I suggest modifying your question a bit so that there isn't any confusion and you can get a proper answer.
Comments.UserDisplayName: Djib2011
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2019-10-30T18:23:15.373Z
Comments.Text: @Djib2011 The current title is the result of my edit, because the previous one was not very descriptive. If you have an idea for a better title, please, edit the question or propose an edit.
Comments.UserDisplayName: nbro
Tags: <recurrent-neural-networks><sequence-modeling>
Title: Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?
UserDisplayName: A_the_kunal
ThreadText: # Why do small datasets require more samples, while big datasets require fewer samples in negative sampling?
  
  (Asked by: username_0 on 2019-10-29)
  
  In the deep learning specialization course by Andrew Ng, in the video [Sequence Models](https://www.coursera.org/lecture/nlp-sequence-models/negative-sampling-Iwx0e) (minute 4:13), he says that in negative sampling we have to choose a sample of words from the corpus to train rather than choosing the whole corpus. But he said that, for smaller datasets, we need a bigger number of samples, for example, 5-20, and, for larger datasets, we need a smaller sample, for example, 2-5. By sample, I am referring to the number of words along with the target word we have taken to train the model.
  
  Why do small datasets require more samples, while big datasets require fewer samples?
  
  **Tags:** recurrent-neural-networks, sequence-modeling
  
  > **Comments:**
  > **username_1**: I think your title is a bit misleading. The way I read it was that you need to have more samples in small datasets, while you are referring to the negative samples in negative sampling (not dataset samples). Also your tags seem a bit irrelevant, especially the one about CNNs. I suggest modifying your question a bit so that there isn't any confusion and you can get a proper answer.  
  > **username_2**: @username_1 The current title is the result of my edit, because the previous one was not very descriptive. If you have an idea for a better title, please, edit the question or propose an edit.  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_3 on 2019-10-30. Score: 1
  
  He likely found this to a be a best practice to avoid over fitting, with a small data set if you only use small and easy to learn (less words -> less degrees of freedom) sequences then you open your model to the risk of over fitting that data set where as on a large data set that has alot more total information you can train on small sequences without being at risk of over fitting because although the smaller sequences will be easier to learn the variance of sequences will be much higher.
  
  
  ---
  
  