Hi,
I have a question about how can I use the Dataset class on my own data.
The problem is that I have my data in several hdf5 files (a medical imaging dataset). Each of these files, has several examples (each one with possibly a different number). If I want to create my own dataset, it says that I should provide the len and getitem methods. However, I dont really have a general index that identifies a sample, I can have an index inside each hdf5 file, but I am not sure how can I use all the data without loading all hdf5 files into memory?
Any suggestion in appreciated.
Thanks
Thanks!