How to set different LOCAL_RANK env variable values for multiple GPUs of a single node machine with Accelerate
This is more of a question than a bug report. 

I am adopting the script with Accelerate from https://github.com/huggingface/transformers/blob/master/examples/pytorch/summarization/run_summarization_no_trainer.py.

I intend to parallel on two gpus of a single machine, one on GPU 0 and the other on GPU 1. As I notice that 
`Line 203:  self.local_process_index = int(os.environ.get("LOCAL_RANK", -1))`
(https://github.com/huggingface/accelerate/blob/209db19dc885887682b07ff88fb6c840cbeb3c1c/src/accelerate/state.py#L128).

Since the LOCAL_RANK env variable is a single integer value (say, I set it to 0), I am just wondering how Accelerate to pick up a second GPU rank (i.e. 1). I don't see that from the above code.

I must have not fully understood the mechanism quite right yet, and haven't found an FAQ of it. Could anyone explain me a bit more.

Thanks
Hi, 

Same problem here, did you find any solution to this problem? @chris-opendata <|||||>Here is the solution:

1. Activate your venv environment.
2. Install accelerate package.
3. Type "accelerate config" at terminal to go through multigpu configuration.
4. In your running bash script, use sth like "accelerate launch main_train.py ...." instead of "python -m main_train.py ..."
    Note that you may not need '&' at the end in order to run it at the background depending on computing env. I am on Slurm.
That is it. Accelerate will work out the rest itself.
