I would load these things in a similar way you load any other sample for your model.
But if these are saved in a file as plain numbers, I would suggest. If it is small enough, do a preprocessing step that reads the whole file, fill in a single Tensor and save that on disk. Then you can just load that Tensor and use it.
Otherwise, store this info next to your input images and load them at the same time as the input image.
In both cases, this should be part of your Dataset so that the Dataloader will reduce any delay in loading it during training.