You could write a custom Dataset and lazily load each sample from your dict in the __getitem__ method.
This tutorial 2 gives you an example on how to write a Dataset.
Let me know, if that works for you.