Oh, that’s trickier.
If I’m not mistaken, PyTorch uses pickle to store and load the data.
It might be possible to load only part of the file via pickle, e.g. by using an io.BytesIO object and seek manually.
However, this sounds like a cumbersome approach and you would also need to dive into the serialization code 3.
Could you load the state_dict once on a machine with enough memory and then store parts of it to use the first approach?