The way the scikit-learn MinMaxScaler works is:
* fit operation: finds the minimum and maximum values of your feature column (mind this scaling is applied separately for each one of your dataframe attributes/columns)
* transform: applies the min max scaling operation, with the values found in the 'fit' operation
Worked example:
let's assumme we have one feature with the next values: [285, 543, 511, 359, 338, 678, 519, 760, 792, 395, 320, 952, 653, 129, 306, 91, 284, 359, 595, 167, 112, 802, 740, 977, 753, 74, 908, 353, 168, 622, 613, 602, 786, 396, 957, 600, 457, 74, 825, 547, 983, 825, 978, 823, 503, 782, 41, 574, 42, 851, 532, 225, 71, 827, 215, 25, 857, 682, 93, 218, 526, 776, 425, 688, 557, 61, 218, 984, 339, 92, 623, 762, 147, 634, 159, 24, 634, 365, 772, 283, 954, 805, 736, 678, 445, 677, 175, 484, 508, 341, 719, 599, 118, 401, 405, 349, 268, 133, 902, 804]
What we need is to rescale each value by applying the min max scaling definition:
rescaled_X_value = (X_value - feature_values_min) / (feature_values_max - feature_values_min)
The point is, what are our min and max values? This is what the 'fit' operation from the scikit-learn MinMaxScaler does with your train data; the point is that once you use this 'fit' operation on train data to find the min and max values, you do not have to repeat it on the test data, you only have to apply the 'transform' operation to just rescale the test data. Let's see how it works:
If we apply it manually:
min_max_scaled_train_data = (train_feature_data - train_feature_data.min())/(train_feature_data.max()-train_feature_data.min())
min_max_scaled_train_data[:10]
The result on the first 10 elements is:
array([0.27111575, <PHONE_NUMBER>, <PHONE_NUMBER>, <PHONE_NUMBER>, <PHONE_NUMBER>,
<PHONE_NUMBER>, <PHONE_NUMBER>, <PHONE_NUMBER>, 0.79979145, <PHONE_NUMBER>])
And with scikit learn scaler:
from sklearn.preprocessing import MinMaxScaler
min_max_scaler = MinMaxScaler()
min_max_scaler.fit(train_feature_data.reshape(-1, 1))
The min_max_scaler has already the info (i.e. min and max values) to be applied on your new data (let's say your test data), without having the fit again. 392
We can also see that the result is the same as doing it manually as above:
min_max_scaler.transform(train_feature_data.reshape(-1, 1))[:10]
array([[0.27111575],
<PHONE_NUMBER>],
<PHONE_NUMBER>],
<PHONE_NUMBER>],
<PHONE_NUMBER>],
<PHONE_NUMBER>],
<PHONE_NUMBER>],
<PHONE_NUMBER>],
[0.79979145],
<PHONE_NUMBER>]])
As an additional check, you can see that the difference between the train_data and all your data (i.e. train + test) when finding the min and max values is not that big, because both come from the same distribution. You can find the example full code here 392