Answers.Body: <p>Steps 1 and 2 are basically the same operation, clipping, which is a possibility but not the best since your loosing information. Imagine having two instances with same features except for feature <span class="math-container">$n$</span>, which assume value 101 and 201 respectively. Let's say that 100 was your observed maximum in the training data for feature <span class="math-container">$n$</span>, after clipping both instances will look the same and lead to same predictions, good for instance 1 cause its value of <span class="math-container">$n$</span> is close to the training maximum, but nonsense for instance 2.</p>
  <p>The best way would be defining a theoretical maximum, so independent from the training data, it could be also a value that we set as maximum and after which we do indeed clip cause occurrences of higher values are rare.</p>
  <p>An alternative is also to use z score standardization instead of min max normalization. The output range is [<span class="math-container">$-\infty$</span>, <span class="math-container">$\infty$</span>] but in practice you'll end up with values almost all within the range [-1, 1], and rare
  occurrences below and above this range are totally fine.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-03T10:13:09.513Z
Answers.Text: If I put let's say a theoretical maximum of 100 for features that usually vary between 0 and 35 and we get a value of 71. In the training, the model will see a max value of .35 but in the prediction, it will get .71. Won`t that mislead the model and create some kind of bias? It's binary classification if that matters.
Answers.UserDisplayName: Newbie
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-03T10:33:11.997Z
Answers.Text: no, the purpose of feature scaling is just to set all features in the same order of magnitude, but is not required that every feature includes instances with values 0 as min and 1 as max after being processed. It's actually extremely rare for that to happen, basically only in computer vision where images are always between 0 and 255.
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-03T10:36:00.043Z
Answers.Text: And also if the variability between training and test is huge, than there is a problem in the quality of the training data that are not representative enough, and that can't be solved with preprocessing.
Answers.UserDisplayName: Edoardo Guerriero
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-03T10:40:55.927Z
Answers.Text: I see thank you. I will actually post a question exactly about the data quality topic soon :).
Answers.UserDisplayName: Newbie
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2022-08-03T09:05:08.250Z
Answers.LastActivityDate: 2022-08-03T09:05:08.250Z
Answers.UserDisplayName: Edoardo Guerriero
Body: <p>So I have this model but the data may vary. And it is virtually impossible to always have the values in bounds. If I do I`d have to use larger period leading to concept shift which is worse.</p>
  <p>The question is what is the best way to deal with the values of futures that are out of the model bounds? I see 3 options</p>
  <ol>
  <li>If the value is greater than max set it to the max value the model has seen</li>
  <li>If the value is less than min set it to the min value the model has seen</li>
  <li>If the value is greater or less set it to the mean that kind of eliminates the future weight for the prediction.</li>
  </ol>
  <p>So what would be the best approach here any thoughts?</p>
  <p>Note: I am retraining the model daily and the model has a lot of futures ~500 so it is highly likely even right after retraining some to be out of bounds, excluding futures is not the best option since it's never the same future showing this behaviour.</p>
  <p>I am using this function for scaling.</p>
  <pre><code>def min_max(value, min_max_map):
      result = 0
      if min_max_map['max'] - min_max_map['min']:
          result = (value - min_max_map['min']) / (min_max_map['max'] - min_max_map['min'])
          if result &gt; min_max_map['max'] or result &lt; min_max_map['min']:
              result = (min_max_map['mean'] - min_max_map['min']) / (min_max_map['max'] - min_max_map['min'])
  
      return result
  </code></pre>
  
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-03T08:23:42.287Z
Comments.Text: is not clear to me why data out of training range represent a problem. Is it cause of feature preprocessing? Are you using min max normalization or something similar?
Comments.UserDisplayName: Edoardo Guerriero
Comments.ContentLicense: CC BY-SA 4.0
Comments.CreationDate: 2022-08-03T08:31:26.770Z
Comments.Text: Yes I am using normalization. As far as I know, models tend to give wrong predictions if the values they are analyzing are never seen before. I am using min max scaler to cramp the values on 1 scale
Comments.UserDisplayName: Newbie
Tags: <deep-learning><deep-neural-networks><prediction><models><features>
Title: How to handle out-of-bound values in Production data?
UserDisplayName: Newbie
ThreadText: # How to handle out-of-bound values in Production data?
  
  (Asked by: username_0 on 2022-08-03)
  
  So I have this model but the data may vary. And it is virtually impossible to always have the values in bounds. If I do I`d have to use larger period leading to concept shift which is worse.
  The question is what is the best way to deal with the values of futures that are out of the model bounds? I see 3 options
  
  If the value is greater than max set it to the max value the model has seen
  If the value is less than min set it to the min value the model has seen
  If the value is greater or less set it to the mean that kind of eliminates the future weight for the prediction.
  
  So what would be the best approach here any thoughts?
  Note: I am retraining the model daily and the model has a lot of futures ~500 so it is highly likely even right after retraining some to be out of bounds, excluding futures is not the best option since it's never the same future showing this behaviour.
  I am using this function for scaling.
  def min_max(value, min_max_map):
      result = 0
      if min_max_map['max'] - min_max_map['min']:
          result = (value - min_max_map['min']) / (min_max_map['max'] - min_max_map['min'])
          if result > min_max_map['max'] or result < min_max_map['min']:
              result = (min_max_map['mean'] - min_max_map['min']) / (min_max_map['max'] - min_max_map['min'])
  
      return result
  
  **Tags:** deep-learning, deep-neural-networks, prediction, models, features
  > **Comments:**
  > **username_1**: is not clear to me why data out of training range represent a problem. Is it cause of feature preprocessing? Are you using min max normalization or something similar?  
  > **username_0**: Yes I am using normalization. As far as I know, models tend to give wrong predictions if the values they are analyzing are never seen before. I am using min max scaler to cramp the values on 1 scale  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2022-08-03. Score: 0 | Accepted answer
  
  Steps 1 and 2 are basically the same operation, clipping, which is a possibility but not the best since your loosing information. Imagine having two instances with same features except for feature $n$, which assume value 101 and 201 respectively. Let's say that 100 was your observed maximum in the training data for feature $n$, after clipping both instances will look the same and lead to same predictions, good for instance 1 cause its value of $n$ is close to the training maximum, but nonsense for instance 2.
  
  The best way would be defining a theoretical maximum, so independent from the training data, it could be also a value that we set as maximum and after which we do indeed clip cause occurrences of higher values are rare.
  
  An alternative is also to use z score standardization instead of min max normalization. The output range is [$-\infty$, $\infty$] but in practice you'll end up with values almost all within the range [-1, 1], and rare
  occurrences below and above this range are totally fine.
  
  > **Comments:**
  > **username_0**: If I put let's say a theoretical maximum of 100 for features that usually vary between 0 and 35 and we get a value of 71. In the training, the model will see a max value of .35 but in the prediction, it will get .71. Won`t that mislead the model and create some kind of bias? It's binary classification if that matters.  
  > **username_1**: no, the purpose of feature scaling is just to set all features in the same order of magnitude, but is not required that every feature includes instances with values 0 as min and 1 as max after being processed. It's actually extremely rare for that to happen, basically only in computer vision where images are always between 0 and 255.  
  > **username_1**: And also if the variability between training and test is huge, than there is a problem in the quality of the training data that are not representative enough, and that can't be solved with preprocessing.  
  > **username_0**: I see thank you. I will actually post a question exactly about the data quality topic soon :).  
  
  ---
  
  