Using large model with fp16 enable causes the server down
I am using a server with Ubuntu 16.04 and 4 TITAN X GPUs. The server runs the base model with no problems. But it cannot run the large model with 32-bit float point, so I enabled fp16, and the server went down.
(When I successfully ran the base model, it consumes 8G GPU memory for each of the 4 GPUS. )
Could you give more informations such as the command that you are using to run the model and the batch size that you are using? Have you tried reducing it?<|||||>Hi, @hguan6 try to adjust the batch size and use gradient accumulation (see [this section](https://github.com/huggingface/pytorch-pretrained-BERT#training-large-models-introduction-tools-and-examples) in the readme and the `run_squad` and `run_classifier` examples) if needed.