How to apply classical multidimensional scaling on new data?
I would be happy if someone gives me a hint for two following questions in Machine Learning - related application of Classical Multidimensional Scaling in Matlab:
1.) It is recommended (but in my experience rarely done) to apply feature extraction (and hence the feature projection methods as Classical Multidimensional or PCA ) separately on the training, validation and test set to get a more realistic model performance estimation during the model validation. Furthermore, I am facing the same issue if I would apply my models after model deployment in a real environment with new data. How can I use the classical multidimensional scaling separately on independent data sets after I have trained my ML model? Do I firstly project my training data on the lower dimension and later provide this determined lower dimensionality to the Classical multidimensional scaling of validation/test/new data?
2.) Is the workflow right: data normalization --> classical multidimensional scaling --> again data normalization on the reduced data? I have provided an example code. I am appreciating every hint! Thank you very much!
Regards, Denys
%% Using of Hold-Out Validation with training, validation and test-set
%% Classical Multidimensional Scaling Training data
DataTrain = normalize(DataTrain); % normalization
D = pdist(DataTrain,'euclidean'); % pair-wise distances
[Y,e] = cmdscale(D); % carry out classical multidimensional scaling
CumSumEig = cumsum(e./sum(e)); % cumulated sum in order to identify the cumlated relevance
LowDim = find(CumSumEig > 0.95, 1, "first"); % dimensions with cumulated relevance over 95 %
DataTrain = Y(:,1:LowDim); % reduced data
%% Validation
% How to reduce the dimensionality of the validation data, because it is
% recommended to apply feature extraction (and hence the feature projection
% methods) separately on the training, validation and test set?
DataValidation = normalize(DataValidation); % normalization
D = pdist(DataValidation,'euclidean'); % pair-wise distances
[Y,e] = cmdscale(D); % carry out classical multidimensional scaling
DataValidation = Y(:,1:LowDim); % reduced data
%% or application to new data
DataNew = normalize(DataNew); % normalization
D = pdist(NewData,'euclidean'); % pair-wise distances
[Y,e] = cmdscale(D); % carry out classical multidimensional scaling
DataNew = Y(:,1:LowDim); % reduced data
% Again Data normalization before making predictions?
DataTrain = normalize(DataTrain);
DataNew = normalize(DataNew); 392