My QUESTION is how run a very big model like bloom on a cluster of machines ?
### System Info

bloom, pytorch, ubuntu

### Who can help?

_No response_

### Information

- [ ] The official example scripts
- [ ] My own modified scripts

### Tasks

- [ ] An officially supported task in the `examples` folder (such as GLUE/SQuAD, ...)
- [ ] My own task or dataset (give details below)

### Reproduction

..

### Expected behavior

Hello i can run opt 66b on one server with 6 gpu 24 Gb by using you page on huggingface on how load big models : I give device_map. I can also run bloom on one server with 8 GPUs 24 GB by giving device_map but it uses offload on CPU and it takes time to answer. My QUESTION is how run a very big model like bloom on a cluster of machines indeed bloom would need 20 GPus 24 Gb and it needs a cluster of 3 machines with 8 gpus to deploy, with accelerate it is not possible as we are limited to only one machine. I have tried everything like to use RPC Framework but it seems it is only for CPU. Thanks for your help. Regards Pat
Please use the [forums](https://discuss.huggingface.co/) for such questions.<|||||>yes thanks for your answer. i wrote it also on forum but it not an easy question and only specialists like you can answer or give me a give me some help so i can continue ... Regards pat<|||||>so could you give some technical help, regards, pat<|||||>This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.

Please note that issues that do not follow the [contributing guidelines](https://github.com/huggingface/transformers/blob/main/CONTRIBUTING.md) are likely to be ignored.