Solved by ptrblck in post #2 
              
              
                You could just derive nn.Module like when creating a model. In the __init__ function you should create your parameters and the forward will perform your computation. 
Since you used just pure PyTorch methods, autograd will be able to create the backward call.