You could try to increase the swap, which would result in a significant slowdown, but you might be able to at least allocate the needed memory.
The better approach would most likely be to reduce the memory usage on this system, e.g. by lowering the batch size.
I’m unsure, if you are preloading the entire dataset, but you might want to lazily load each sample in this case.