Autograd will use the backward methods of each submodule used in the nn.Transformer.forward method 1 to calculate the gradients so the nn.Transformer module doesn’t necessarily need to implement a custom backward method.
You could check these submodules and see, how the backward methods are defined (i.e. if they are using custom ones or just other PyTorch operations with already defined backwards).