Thanks @ptrblck for your thoughts. I am using the same weights i.e., loaded weights of TensorFlow in PyTorch. I have set everything else as same but the last thing I was thinking of towards padding. What I have read about padding is that TensorFlow uses asymmetric whereas PyTorch uses symmetric. PyTorch has provided same padding in version 1.9.0 but I am using 1.4.0. Can this be an issue?