Hi,
It varies a lil depending on some variables.
The main idea is that mmap allow you to set an instance of a numpy array without reading its content.
The content is read once you slice the array. This is obviously super efficient to read data from large arrays/tensors.
There are some variations you can do depending on two things, how many samples you have and how large they are.
The most efficient one is to create a large array by stacking all your samples (you may need to pad if they are different length).
Then you instantiate this mmap in the init function and slice it (read it) in the getitem.
If you have hundreds of files you can save them independently, instantiate the mmap in the init and read them in the getittem.
If you ahve millions of files, you can save them independently but you would probably need to instantiate the mmap and read in getitem.
This is really up to you.
All you need to do is to make an algorithm which predifines which samples and segments you are going to read, ensuring you use your data as much as possible.
From this you can infer the length of the dataset and code the getitem.