good insight!
may i ask how to ’ initialize total_features with copies of features’ exactly? simply use copy.deepcopy will cause error…
RuntimeError("Only Tensors created explicitly by the user (graph leaves) support the deepcopy protocol at the moment"

another question is, does all the distributed operators not record autograd? like dist.all_reduce and so on… where can i get information about these characteristics (record or not)? i didn’t see that in official doc 1
thank you!