The UnpicklingError error might be raised, if pickle isn’t able to load a specific fine, if it the binary file is corrupt (e.g. due to a failed download).
If that’s the case, disabling shuffle and rerunning the script should yield the error at the same index.
However, based on the previous error message (OSError: [Errno 22] Invalid argument) it seems as if a wrong path was used to try to load a file.