`device_map="auto"` support multi-node
### Feature request

`AutoModel.from_pretrained(model_dir, device_map="auto", trust_remote_code=True).half()`
I want to load a huge model in multi-node for inference, such as 4 node with 1 gpu per node. But I do not know how to do it. The `device_map="auto"` seems only work for one node.

### Motivation

I want to test the long-context ppl. When I increase the context, the gpu memory increase too. So I need more node to do the inference.

### Your contribution

not yet.
cc @younesbelkada 

Hi @guozhiyao, 

Thanks for raising this issue. At the moment, there isn't enough information for us to be able to help you. Could you specify what you mean by "doesn't work"? 

As a side note, I don't think you can do `device_map=auto"` and then `.half()`, instead you can pass in a flag to `from_pretrained` to specify the precision e.g. `torch_dtype=torch.float16` or `load_in_8bit=True`. @younesbelkada  can confirm :) 



<|||||>hi @guozhiyao 
thanks for raising this up ! 
firstly as @amyeroberts suggested, the canonical way of loading a model with a specific dtype (in your case `half`=`torch.float16`) is by passing `torch_dtype=torch.float16` thus you avoid any unexpected issue you may encounter

Regarding your second question, I don't think this is supported by `device_map="auto"` for inference. Usually the multi-node paradigm is useful for training, where you have an entire training process running independently on each node. I think accelerate supports multi-node training (you can select mutli node training when running `accelerate config` and we have made some training process work under multi-node regime using accelerate internally). 

However I doubt that you can run multi-node inference out of the box with `device_map='auto'` as this is intended only for single node (single / multi GPU or CPU only). In multi-node setting each process will run independently `AutoModel.from_pretrained(model_dir, device_map="auto", trust_remote_code=True).half()` thus the model will not be shared across both processes.
I am also unsure about the benefits of such protocol - the only case it might be interesting to see this would be if someone wants to fit a model that can't fit in more than a node (more than 8xA100 80GB at most)

Would like also to hear from @sgugger or @muellerzr , in case I missed something I am not aware of.<|||||>I can confirm that multi-node is not supported by `device_map="auto"` :-) 