@Arnab_Mondal check the suggestions I shared in my comments. They are likely gonna help with that. For SimSiam the following factors collectively contribute to preventing the representation collapse:

stop_gradient()
Augmentations
The specific use of Batch Norm
The use of an Autoencoder like predictor network

All of these are reflected in the initial example I posted barring augmentations and the exact number of dense units in the predictor.
As I mentioned previously in my comments, I am happy to help here with all the suggestions I can provide from my little experience of working with SSL. So, please keep this thread posted with anything you find.