RAG : Can we fine-tune RAG with update frequency method similar to Fairseq framework?
RAG fine-tune script needs 8 GPUs to train. Is there any chance that the training can be done with less number of GPUs using the update frequency? 
Hey @shamanez - what do you mean by "update frequency"? You don't need to use 8 GPUS => you can just reduce the number of gpus as you wish and keep the "same" batch size by increasing the `gradient_accumulation_steps` - does this make sense? <|||||>Let's say if the effective batch size is 32 with 8 GPUs and I want to keep the same batch size with 4 GPUs, I just need to change the _gradient_accumulation_steps_ to 2 right?

[Update_Freq](https://fairseq.readthedocs.io/en/latest/command_line_tools.html#fairseq-train) is what fairseq used to keep the effective batch size same with less number of GPUs.<|||||>Yeah exactly, in `examples/rag/finetune.sh` the default is `gpus=8` and `gradient_accumalation_steps=1`. So if you want to use less gpus while keeping the same "effective" batch size you should increase `gradient_accumalation_steps` accordingly<|||||>Thanks a lot. :) 