The number of parameters in both pytorch and tensorflow versions are 14991. I removed the fc layers from tensorflow implementation (only has conv2d and transpose conv2d layers).
I also removed cudnn.deterministic = True. I literally have only conv2d and transpose conv2d layers in both.
I have included my main function in the code above. Do you think some unwanted broadcasting is happening? I don’t think there is anything wrong with this pytorch code.