Answers.Body: <p>Scaling individual parts of a dataset individually is generally not that trivial. Just remember that this means that the statistical properties of the subsets also may change. E.g., let's assume that your model just takes the mean of all sub-datasets. To combine the information from each sub-dataset you could simply averaging over all means. However, if the sub-datasets were differently scaled, averaging over them is not reasonable. The same holds for ML models.</p>
  <p>However, if you are careful and properly scale the results of each sub-dataset then this might make perfectly sense.</p>
  <p>To answer your last question: yes, scaling the entire dataset e.g. to (-1, 1) or any other reasonable range would probably be the easiest approach here. If this doesn't work due to too small difference then you still could think about non-linear scaling.</p>
  
Answers.ContentLicense: CC BY-SA 4.0
Answers.CreationDate: 2024-04-07T13:06:48.833Z
Answers.LastActivityDate: 2024-04-21T14:10:26.453Z
Answers.LastEditDate: 2024-04-21T14:10:26.453Z
Answers.UserDisplayName: BanDoP
Body: <p>I am currently making an LSTM model to predict the change in value of a stock for each day. My dataset consists of 4 years of data for 30 different stocks along with some financial metrics such as RSI, EMA... This data is concatenated with a few macroeconomic indicators and some market interest indicators obtained from google trends.
  For the LSTM model I have scaled the data between -1 and 1. However, because the data for each stock is so different (some stocks are valued at close to 1<span class="math-container">$ per share whereas others trade at $</span>600 per share), I have scaled the data on a stock by stock basis using the min, max scaler. The issue I have is I don't know how to inverse the scale to get usable results for comparison?
  Should I have scaled the entire dataset as one even though this will lead to some stocks having almost not difference from one day to the next?</p>
  
Tags: <long-short-term-memory><datasets><data-preprocessing><data-science>
Title: Can I scale subsets of my dataset independently to handle different feature ranges?
UserDisplayName: Daniel goldney
ThreadText: # Can I scale subsets of my dataset independently to handle different feature ranges?
  
  (Asked by: username_0 on 2024-04-06)
  
  I am currently making an LSTM model to predict the change in value of a stock for each day. My dataset consists of 4 years of data for 30 different stocks along with some financial metrics such as RSI, EMA... This data is concatenated with a few macroeconomic indicators and some market interest indicators obtained from google trends.
  For the LSTM model I have scaled the data between -1 and 1. However, because the data for each stock is so different (some stocks are valued at close to 1$ per share whereas others trade at $600 per share), I have scaled the data on a stock by stock basis using the min, max scaler. The issue I have is I don't know how to inverse the scale to get usable results for comparison?
  Should I have scaled the entire dataset as one even though this will lead to some stocks having almost not difference from one day to the next?
  
  **Tags:** long-short-term-memory, datasets, data-preprocessing, data-science
  
  
  ---
  
  ## 1 answers
  
  ### Answer by username_1 on 2024-04-07. Score: 1 | Accepted answer
  
  Scaling individual parts of a dataset individually is generally not that trivial. Just remember that this means that the statistical properties of the subsets also may change. E.g., let's assume that your model just takes the mean of all sub-datasets. To combine the information from each sub-dataset you could simply averaging over all means. However, if the sub-datasets were differently scaled, averaging over them is not reasonable. The same holds for ML models.
  
  However, if you are careful and properly scale the results of each sub-dataset then this might make perfectly sense.
  
  To answer your last question: yes, scaling the entire dataset e.g. to (-1, 1) or any other reasonable range would probably be the easiest approach here. If this doesn't work due to too small difference then you still could think about non-linear scaling.
  
  
  ---
  
  