The training isn’t distributed and torch.distributed isn’t used.
By distributed I mean the workers used to collect data are distributed and the network params are send from trainer to these workers through mp.queue.
Once the data are collected and trainer starts to train, the workers stop working so I suppose there’s no interaction between the workers and the trainer. So what appears really strange to me is that the backward is done but step is not.
I’m using the standard optim.Adam