![image](https://user-images.githubusercontent.com/41872440/216619913-99568f8c-8a82-42ab-ae7a-dd56846dd395.png)
![image](https://user-images.githubusercontent.com/41872440/216619913-99568f8c-8a82-42ab-ae7a-dd56846dd395.png)
I have 8 gpu's in this machine.
![image](https://user-images.githubusercontent.com/41872440/216619996-7af22496-7fb8-4b6d-90f2-3606846b3cdc.png)
I think its not taking all 8 gpu's. Already tried changing batch_sizes and with multiple of 8

_Originally posted by @namanpundir in https://github.com/huggingface/transformers/issues/21407#issuecomment-1415898415_
            
Please do not spam the repository by opening duplicate issues. You can find help to debug your training by posting on the [forums](https://discuss.huggingface.co/) as we keep issues for bugs in the library and feature requests only. You will need to share how you are launching your script for anyone to be able to help.