William_Richards:

The issue I currently have is constructing a custom data structure that deals with many images to one label


I’m not sure if you really want to design the Dataset to return multiple images and a single target or if you rather want to repeat the target for all images in the same interval.
Both approaches would be possible by writing a custom Dataset as described in this tutorial 1.
Depending how the targets are stored, you might want to create them in the Dataset.__init__ method and lazily load the data (and index the target) in __geittem__.



 William_Richards:

as well as allowing for the desired cross-validation.


I would recommend to use some methods from sklearn.model_selection and split the dataset indices according to your groups. E.g. GroupKFold might be useful.
Once you have these split indices, you could wrap your datasets into Subsets with these indices and perform your cross validation.