How did you apply the normalization on the raw values?
By raw values do you mean the input or the features from the sub-models?
For the pre-training strategy. Did you save the sub-model outputs or freeze the sub-models while training the second stage model?
Could you provide the sub-model features as Tensors so that I could have a look?
How was the accuracy of both sub-models separately?