torchvision really only uses 8-bit RGB as an intermediate format, as images are usually converted to floating point before being passed to a model with something like to_tensor() 1.
You can also see a typical example of how transformations are applied before being passed to a model in the ImageNet example:
  
      github.com
  
  
    pytorch/examples/blob/cbb760d5e50a03df667cdc32a61f75ac28e11cbf/imagenet/main.py#L207 2

        print("=> no checkpoint found at '{}'".format(args.resume))


cudnn.benchmark = True


# Data loading code
traindir = os.path.join(args.data, 'train')
valdir = os.path.join(args.data, 'val')
normalize = transforms.Normalize(mean=[0.485, 0.456, 0.406],
                                 std=[0.229, 0.224, 0.225])


train_dataset = datasets.ImageFolder(
    traindir,
    transforms.Compose([
        transforms.RandomResizedCrop(224),
        transforms.RandomHorizontalFlip(),
        transforms.ToTensor(),
        normalize,
    ]))


if args.distributed:
    train_sampler = torch.utils.data.distributed.DistributedSampler(train_dataset)



  
  
    
    
  
  


Loading a floating point image should not be an issue. If for some reason you need to use a transform that only works on PIL Images, you can use the to_pil_image 3 function.