mrshenli:

the GPU memory size needs to be at least 3X of the model size.


Thanks for your reply! I have specified the cuda ids when training. When I trained in a single GPU, it returns the error message that ‘CUDA requires ~6G more memory’. I add one more 16G GPU to do the training. According to your suggestion, it seems like I need at least 4 GPUs in total? Thanks!