Okay. Got it!!
They don’t apply the ToTensor and Normalize to the masks even though they are passed into the functions!  I was going crazy over this!
Here the two snippets of code, that brought me back from crazy
In this file, here 5

  
      github.com
  
  
    pytorch/vision/blob/3c81d474a2525b11356ca1dde299a9b1ab3715c6/references/segmentation/transforms.py#L80-L82 6

image = F.to_tensor(image)
target = torch.as_tensor(np.asarray(target), dtype=torch.int64)
return image, target



  
  
    
    
  
  



  
      github.com
  
  
    pytorch/vision/blob/3c81d474a2525b11356ca1dde299a9b1ab3715c6/references/segmentation/transforms.py#L91-L92 6

image = F.normalize(image, mean=self.mean, std=self.std)
return image, target



  
  
    
    
  
  


See, the transforms are never applied to the masks! YAY!
Happy Coding Everyone!