What doesn’t really make sense, is that the torch.cat operation apparently needs more memory in the 20th epoch, whereas before that it seems to work.
Model sharding would be an approach to split your model onto both GPUs without sacrificing the model capacity.
Since I still think there might be an issue with your code, could you please post it?
You can add code by wrapping it in three `.