Thank you for your quick reply. I am well aware that tutorials are not meant to be benchmark implementations but I was surprised that it did not train usable representations at all.
By true random cropping I mean a random crop and resize as mentioned in the paper. This crops a random percentage of the original image, given as hyperparameters, and then resizes the cropped image to the original size. “tf.image.random_crop” as used in the tutorial, randomly crops an image at the size given, since we’re just cropping a 32x32 image to 32x32 it is effectively doing nothing. I thought that the lack of cropping was causing the tutorial code to not work, but fixing this did not improve the performance.
I did not match the backbone from the paper and their hyperparameters. I used what you used in the tutorial as it is very close. I’ll try changing the backbone and maybe it’ll help but I’d be very surprised if a slightly different backbone changes the linear transfer score from 35% to the 90% reported in the paper. It would also be pretty damning for SimSiam if it turned out the backbone was the primary contributor to benchmark success.
Thanks for any help. I’m just trying to get a clean, simple, implementation that performs roughly similar to the paper and hopefully find a few tweaks that could allow a person to get a usable trained model from the tutorial.