How can u make sure that my transformer model should only one GPU, though the serve has multiple GPU cards.
I have transformer BERT model and I am trying to train on lambda server which has 8 GPU cards, How can u make sure that this model should use only once GPU out of 8, by default , it is using all GPUs. even after setting 
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

You should have used the template, because now we don't have enough information to help you: how are you running the script (torch launch utility? Which command?), which script are you using (your own (give details) or one of the example scripts)?

By default, PyTorch will only use one GPU unless you specify it to go DDP.<|||||>Use tags please. Read through this guide. https://help.github.com/en/github/writing-on-github/creating-and-highlighting-code-blocks<|||||>@BramVanroy  it was my bad, I am in bit hurry so that  I was not able to provide my code path. I have just pushed my code to github. Please check below for path 
[https://github.com/tiru1930/bert_intent_classification](url)

In this PATH I use src/Train.py to train the model.

<|||||>As you see, you are just wasting your own time and mine when you are _in a hurry_. In the future, take your time to write a good starting posts so that we _want_ to help you and _can_ help you quickly.

In your code, you are calling DataParallel on your mode, which will automatically run your model over multiple GPU's (but under a single process). Remove this line.

https://github.com/tiru1930/bert_intent_classification/blob/master/src/train.py#L80