Ah so a single batch can be done in memory, but the maximum batch size for the data loader is only 4, which we feel is too small. However, if we instead treat each batch as a “sub-batch” (idk the proper term) then we can stich 8 of these together to emulate a batch size of 32. So we want to accumulate gradients across iterations of the data loader.
In any case the code now works, thank you so much!