11123:

I found there is a parameter self.next_backprop in your code, do I need to set it? Sorry to bother you again.





 Andreas_Georgiou:

diffdist.functional.all_gather(gather_list, tensor)


Apologies, the line should be
gather_list = diffdist.functional.all_gather(gather_list, tensor)`

If you get any errors try setting inplace=False. No need to use next_backprop