If I understand your use case correctly, you are loading a .npy file for each sample? If so, I don’t think it’s a bad idea or do you have any specific concerns?
I think the usual approach of loading image types (e.g. JPEG) has given you the advantage to apply image transformations on the samples directly, but since torchvision.transforms now also supports transformations on tensors you could also load the arrays/tensors directly.