Still crashes after 3 iterations.
Used htop and nvidia-smi to see where the memory problem was, and its definitely in the gpu. Its wierd cause if I dont use my data_loader function, and instead train over the same batch over and over manually, the model trains just fine.