I trained with older version of Pytorch on workstation
# On Workstation  -- older version
torch              1.6.0
torchvision        0.7.0

Then copied it to Nvidia-hardware.
# On Nvidia-ProAI  -- Newer version
torch              1.7.0a0
torchvision        0.8.0a0+2f40a48

Still the same problem or error persists.
Problem seems to be similar to this post:  https://discuss.pytorch.org/t/assertion-error-when-using-resnet-18-as-backbone-with-faster-rcnn/113965.