Thanks for the reply. Yes, that seems to be the case. I converted the target_size to torch tensor before passing it to upsample function so that kinda solved the problem.
From this
x7 = self.conv7(x6)
# UPSAMPLE
 up = self.upsample1(x7, downsample4.size())

to
x7 = self.conv7(x6)
# UPSAMPLE
featuremap_size = torch.tensor(downsample4.size())
up = self.upsample1(x7, featuremap_size, self.inference)

But the model that I am trying to optimize is YoloV4 and it has some activation functions(mish and softplus) not supported on pytorch’s quantization. Therefore, even after this solution, I was not able to quantize the model in the end.