I’m only using 2313MiB of GPU RAM (out of 24GB) with a batch_size of 8192. If I set increase my batch_size to 16384, then sometimes it randomly crashes.
As for my model and training samples, I have:
Total params: 10,562,592
Trainable params: 10,562,592
Non-trainable params: 0
----------------------------------------------------------------
Input size (MB): 0.00
Forward/backward pass size (MB): 0.32
Params size (MB): 40.29
Estimated Total Size (MB): 40.62
----------------------------------------------------------------
Training Samples:       28975661
Validation Samples:     1552227