Body: <pre><code>from zlib import crc32
  
  def is_id_in_test_set(identifier, test_ratio):
      return crc32(np.int64(identifier)) &lt; test_ratio * 2**32
  
  def split_data_with_id_hash(data, test_ratio, id_column):
      ids = data[id_column]
      in_test_set = ids.apply(lambda id_: is_id_in_test_set(id_, test_ratio))
      return data.loc[~in_test_set], data.loc[in_test_set]
  
  
  
  housing_with_id = housing.reset_index()  # adds an `index` column
  train_set, test_set = split_data_with_id_hash(housing_with_id, 0.2, &quot;index&quot;)
  </code></pre>
  <p>book says</p>
  <blockquote>
  <p>compute a hash of each instance’s identifier and put that instance in the test set if the hash is lower than or equal to 20% of the maximum hash value. This ensures that the test set will remain consistent across multiple runs, even if you refresh the dataset. The new test set will contain 20% of the new instances, but it will not contain any instance that was previously in the training set.</p>
  </blockquote>
  <p>I don't understand how if the identifier is less than 20% of the maximum hash value will guarantee that an instance does not repeat in the test set</p>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-30T21:45:00.837Z
Comments.Text: Could you clarify what you mean by "repeat"? Note the quoted text doesn't use that word, or make any claims about data repetition. It makes claims about instances with ids. It also doesn't say anything about comparing an id with a hash.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-31T15:11:20.500Z
Comments.Text: @NeilSlater The new test set will contain 20% of the new instances, but it will not contain any instance that was previously in the training set.
Comments.UserDisplayName: samsamradas
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-31T15:33:16.803Z
Comments.Text: OK, that's the claim in the the quote from the book. But what do you mean by the word *repeat* in the phrase "does not repeat in the test set"? What would it mean for an instance to "repeat in the test set"? I am reasonably confident that I understand what the book is trying to say in the quote. I am **not** confident that I understand what your misunderstanding is, if that makes sense . . . and that's because you are using this word "repeat", and I am not sure what you mean by an instance "repeating". Normally when an item repeats, it means it appears twice or more
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-01-31T22:03:43.587Z
Comments.Text: That makes sense, but as each instance only appears once, and is either in the test set or the training set, it is not surprising, and is not really related to what the book is claiming.
Comments.UserDisplayName: Neil Slater
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2024-02-01T12:29:54.763Z
Comments.Text: sorry an instance can't be on the test set if it was on the test set already
Comments.UserDisplayName: samsamradas
Tags: <machine-learning><testing><validation-datasets>
Title: I dont understand this way of having a stable train/test split even after updating the dataset
UserDisplayName: samsamradas
ThreadText: # I dont understand this way of having a stable train/test split even after updating the dataset
  
  (Asked by: username_0 on 2024-01-30)
  
  ```
  from zlib import crc32
  
  def is_id_in_test_set(identifier, test_ratio):
      return crc32(np.int64(identifier)) < test_ratio * 2**32
  
  def split_data_with_id_hash(data, test_ratio, id_column):
      ids = data[id_column]
      in_test_set = ids.apply(lambda id_: is_id_in_test_set(id_, test_ratio))
      return data.loc[~in_test_set], data.loc[in_test_set]
  
  housing_with_id = housing.reset_index()  # adds an `index` column
  train_set, test_set = split_data_with_id_hash(housing_with_id, 0.2, "index")
  
  ```
  
  book says
  
  > compute a hash of each instance’s identifier and put that instance in the test set if the hash is lower than or equal to 20% of the maximum hash value. This ensures that the test set will remain consistent across multiple runs, even if you refresh the dataset. The new test set will contain 20% of the new instances, but it will not contain any instance that was previously in the training set.
  
  I don't understand how if the identifier is less than 20% of the maximum hash value will guarantee that an instance does not repeat in the test set
  
  **Tags:** machine-learning, testing, validation-datasets
  
  > **Comments:**
  > **username_1**: Could you clarify what you mean by "repeat"? Note the quoted text doesn't use that word, or make any claims about data repetition. It makes claims about instances with ids. It also doesn't say anything about comparing an id with a hash.  
  > **username_0**: @username_1 The new test set will contain 20% of the new instances, but it will not contain any instance that was previously in the training set.  
  > **username_1**: OK, that's the claim in the the quote from the book. But what do you mean by the word *repeat* in the phrase "does not repeat in the test set"? What would it mean for an instance to "repeat in the test set"? I am reasonably confident that I understand what the book is trying to say in the quote. I am **not** confident that I understand what your misunderstanding is, if that makes sense . . . and that's because you are using this word "repeat", and I am not sure what you mean by an instance "repeating". Normally when an item repeats, it means it appears twice or more  
  > **username_1**: That makes sense, but as each instance only appears once, and is either in the test set or the training set, it is not surprising, and is not really related to what the book is claiming.  
  > **username_0**: sorry an instance can't be on the test set if it was on the test set already  
  
  ---
  
  ## 0 answers
  
  