Currently the division by 255. is done, when the transformation assumes an image.
Using a np.array this is done by checking against the dtype.
If you pass a np.uint8 array into the transformation, it will be scaled.
You can find the line of code here 118.
data = np.array(np.random.rand(3, 3, 1) * 255., dtype=np.uint8)
transforms.ToTensor()(data)