Solved by InnovArul in post #2 
              
              
                Ideally, yes. you are right. If you do not need any custom functions for your model, you need not use autograd.Function. You could just use modules from torch.nn packages, APIs from  torch, torch.nn.functional packages. As you said, you can create normal python function with these predefined APIs fr…