Body: <p>The acronym &quot;iid&quot; stands for &quot;independent and identically distributed&quot;. It is a <strong>property of a sequence of random variables</strong>. You can <a href="https://www.statlect.com/glossary/IID-sequence" rel="nofollow noreferrer">read here</a> for more details. This question is just about the usage of the word &quot;iid&quot; in contemporary machine learning and is not about the feasibility of checking iid based on either associated joint distribution or dataset.</p>
  <p>In the formal and strict sense, the word &quot;iid&quot; should be used only as a property for a sequence of random variables based on the underlying joint probability distribution function. But, I noticed that there is another (maybe less-strict) usage for the word 'iid' based on the context.</p>
  <p>Consider the following statements compiled from different answers to my questions <a href="https://ai.stackexchange.com/questions/31654/how-can-we-draw-i-i-d-from-any-probability-distribution">1</a>,<a href="https://ai.stackexchange.com/questions/31658/which-of-the-following-probability-distribution-is-generating-an-iid-dataset">2</a></p>
  <p>From <a href="https://ai.stackexchange.com/a/31655/18758">this</a> answer</p>
  <blockquote>
  <p>The term <strong>i.i.d. is a property of a dataset</strong>. A dataset can be created
  that is i.i.d. with respect to a particular probability distribution.
  It doesn't matter what that distribution is, it just has to exist, and
  be relevant to the purpose the ML is being put to.</p>
  </blockquote>
  <p>From <a href="https://ai.stackexchange.com/a/31678/18758">this</a> answer</p>
  <blockquote>
  <p>The point is even you know the distribution, sometimes you can't prove
  that the <strong>sampled data is i.i.d. or not</strong>!....</p>
  </blockquote>
  <p>From <a href="https://ai.stackexchange.com/a/31681/18758">this</a> answer</p>
  <blockquote>
  <p>....A <strong>table of results of dice throws is likely iid</strong>...... (there are some issues with this answer, but the bolded excerpt is true)</p>
  </blockquote>
  <p>So, the usage of the word iid, in this sense, is somewhat different. Although I think, iid is a property of a sequence of random variables in this sense also, it is okay to use the word 'iid' for a dataset (collection of samples) since the dataset represents some underlying probability distribution.</p>
  <p>Thus, the two usages I am aware of up to now are</p>
  <ol>
  <li><p>iid for a sequence of random variables based on joint distribution.</p>
  </li>
  <li><p>iid for a sequence of random variables based on the collection of samples.</p>
  </li>
  </ol>
  <p>Is my understanding of the two usages of the word &quot;iid&quot; correct? and are there any other usages for the word &quot;iid&quot;?</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-14T07:43:18.927Z
Comments.Text: @NeilSlater what about the bolded portion?
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-14T07:46:58.750Z
Comments.Text: "[A] table of results of dice throws is likely iid" seems correct to me (with reasonable assumptions). However, the non-bolded "it is because the dice roll itself is iid" is not correct
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-14T08:10:33.500Z
Comments.Text: Why do you get hung up on definitions of words so much? The important thing is the concept, not the association between words and concepts. Different people may use the same word with slightly different definitions, and that's fine.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-14T08:25:50.990Z
Comments.Text: @user253751 It may be due to self-study. I am trying to understand underlying concepts. Unfortunately, these terminological issues are always popping up for me.
Comments.UserDisplayName: hanugm
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-14T08:30:57.643Z
Comments.Text: consider that a sample *is* a random variable, in a way .When you sample random variable X 5 times it is the same as sampling from 5 random variables whose distribution is identical to the distribution of X and which are independent.
Comments.UserDisplayName: Criticize SE actions means ban
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2021-09-14T08:32:30.807Z
Comments.Text: Oh! seems to be intuitive. I will think on it... Then, it may be similar to unbiased sampling. @user253751
Comments.UserDisplayName: hanugm
Tags: <machine-learning><terminology><datasets><random-variable><iid>
Title: What are the different possible usages of the word "i.i.d" in machine learning?
UserDisplayName: hanugm
ThreadText: # What are the different possible usages of the word "i.i.d" in machine learning?
  
  (Asked by: username_0 on 2021-09-14)
  
  The acronym "iid" stands for "independent and identically distributed". It is a **property of a sequence of random variables**. You can [read here](https://www.statlect.com/glossary/IID-sequence) for more details. This question is just about the usage of the word "iid" in contemporary machine learning and is not about the feasibility of checking iid based on either associated joint distribution or dataset.
  
  In the formal and strict sense, the word "iid" should be used only as a property for a sequence of random variables based on the underlying joint probability distribution function. But, I noticed that there is another (maybe less-strict) usage for the word 'iid' based on the context.
  
  Consider the following statements compiled from different answers to my questions [1](https://ai.stackexchange.com/questions/31654/how-can-we-draw-i-i-d-from-any-probability-distribution),[2](https://ai.stackexchange.com/questions/31658/which-of-the-following-probability-distribution-is-generating-an-iid-dataset)
  
  From [this](https://ai.stackexchange.com/a/31655/18758) answer
  
  > The term **i.i.d. is a property of a dataset**. A dataset can be created
  > that is i.i.d. with respect to a particular probability distribution.
  > It doesn't matter what that distribution is, it just has to exist, and
  > be relevant to the purpose the ML is being put to.
  
  From [this](https://ai.stackexchange.com/a/31678/18758) answer
  
  > The point is even you know the distribution, sometimes you can't prove
  > that the **sampled data is i.i.d. or not**!....
  
  From [this](https://ai.stackexchange.com/a/31681/18758) answer
  
  > ....A **table of results of dice throws is likely iid**...... (there are some issues with this answer, but the bolded excerpt is true)
  
  So, the usage of the word iid, in this sense, is somewhat different. Although I think, iid is a property of a sequence of random variables in this sense also, it is okay to use the word 'iid' for a dataset (collection of samples) since the dataset represents some underlying probability distribution.
  
  Thus, the two usages I am aware of up to now are
  
  1. iid for a sequence of random variables based on joint distribution.
  2. iid for a sequence of random variables based on the collection of samples.
  
  Is my understanding of the two usages of the word "iid" correct? and are there any other usages for the word "iid"?
  
  **Tags:** machine-learning, terminology, datasets, random-variable, iid
  > **Comments:**
  > **username_0**: @NeilSlater what about the bolded portion?  
  > **username_1**: "[A] table of results of dice throws is likely iid" seems correct to me (with reasonable assumptions). However, the non-bolded "it is because the dice roll itself is iid" is not correct  
  > **username_2**: Why do you get hung up on definitions of words so much? The important thing is the concept, not the association between words and concepts. Different people may use the same word with slightly different definitions, and that's fine.  
  > **username_0**: @user253751 It may be due to self-study. I am trying to understand underlying concepts. Unfortunately, these terminological issues are always popping up for me.  
  > **username_2**: consider that a sample *is* a random variable, in a way .When you sample random variable X 5 times it is the same as sampling from 5 random variables whose distribution is identical to the distribution of X and which are independent.  
  > **username_0**: Oh! seems to be intuitive. I will think on it... Then, it may be similar to unbiased sampling. @user253751  
  
  ---
  
  ## 0 answers
  
  