You are detaching the computation graph here:
p = torch.argmax(p, dim=1).type(torch.float32).requires_grad_(True)

torch.argmax is not differentiable and you would have to use the model outputs to calculate the loss.