Solved by ptrblck in post #2 
              
              
                Autograd will use the backward methods of each submodule used in the nn.Transformer.forward method to calculate the gradients so the nn.Transformer module doesn’t necessarily need to implement a custom backward method. 
You could check these submodules and see, how the backward methods are defined (…