weights should perform similar to the weights for multi modality models. what i require is that at the classification layer after concatenation, should know that layer has to focus more on the features extracted from the left side branch