Answers.Body: <p>The sentences coming from the same document, author, etc., are unlikely to be <a href="https://en.wikipedia.org/wiki/Independence_(probability_theory)" rel="nofollow noreferrer">independent</a>, that is, the occurrence of a sentence <span class="math-container">$s_i$</span> in a certain document <span class="math-container">$d$</span> is likely correlated with the occurrence of another sentence <span class="math-container">$s_j$</span>. If they are not independent, they can also not be independent and identically distributed (which is a stronger condition). The same can be said for words in the same sentence. </p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2020-03-06T21:10:45.937Z
Answers.LastActivityDate: 2020-03-06T21:10:45.937Z
Answers.UserDisplayName: nbro
Body: <p>I am trying to build an LSTM model to generate Shakspeare-like poems. I have data set <span class="math-container">$\{s_1, s_2, \dots, s_m \}$</span>, which are sentences of Shakespeare poems, and each sentence contains words <span class="math-container">$\{w_1, w_2, \dots, w_n \}$</span>. </p>
  
  <p>I am wondering:  Are different <span class="math-container">$s_i$</span> (<span class="math-container">$i=1, \dots, m$</span>) independent and identically distributed samples (IID)? Are <span class="math-container">$w_i$</span> (<span class="math-container">$i=1, \dots, n$</span>) within each sentence the IID?</p>
  
Tags: <machine-learning><datasets><long-short-term-memory><cross-entropy>
Title: Are sentences from the same document independent and identically distributed?
UserDisplayName: Leey
ThreadText: # Are sentences from the same document independent and identically distributed?
  
  (Asked by: username_0 on 2020-03-06)
  
  I am trying to build an LSTM model to generate Shakspeare-like poems. I have data set $\{s\_1, s\_2, \dots, s\_m \}$, which are sentences of Shakespeare poems, and each sentence contains words $\{w\_1, w\_2, \dots, w\_n \}$.
  
  I am wondering: Are different $s\_i$ ($i=1, \dots, m$) independent and identically distributed samples (IID)? Are $w\_i$ ($i=1, \dots, n$) within each sentence the IID?
  
  **Tags:** machine-learning, datasets, long-short-term-memory, cross-entropy
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2020-03-06. Score: 3 | Accepted answer
  
  The sentences coming from the same document, author, etc., are unlikely to be [independent](https://en.wikipedia.org/wiki/Independence_(probability_theory)), that is, the occurrence of a sentence $s\_i$ in a certain document $d$ is likely correlated with the occurrence of another sentence $s\_j$. If they are not independent, they can also not be independent and identically distributed (which is a stronger condition). The same can be said for words in the same sentence.
  
  
  ---
  
  