LengthGroupedSampler: why longest examples go first?
Is there any reason for the longest, but not the shortest, examples to go first? I think in some curriculum learning literature people start with the easiest example. What's design decision behind starting with the longest ones?
Correct me if I'm wrong @sgugger, but I believe this is because this way, if a memory error happens, it's at the absolute beginning - and not mid-training, which is much more annoying.<|||||>Exactly, this is to get the OOM as soon as possible if it should ever happen.<|||||>That makes a lot of sense -- thanks!