I think you are just losing the accuracy due to quantization.
If you transform your FloatTensor to a PIL.Image, it will be scaled to [0, 255] in uint8 type.
This will already quantize your values, as you cannot map all floating values in [0, 1] to a ByteTensor.
The reverse (ToTensor()) thus yields a FloatTensor with these already quantized values.